# High memory usage when running multiple ansible playbooks on localhost **URL:** \ **Category:** Ansible Project\ **Created:** [December 12, 2018, 6:53am UTC](https://forum.ansible.com/t/high-memory-usage-when-running-multiple-ansible-playbooks-on-localhost/29244 "2018-12-12T06:53:50Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![pradeep.hk](https://avatars.discourse-cdn.com/v4/letter/p/8c91f0/32.png) [@pradeep.hk](https://forum.ansible.com/u/pradeep.hk)\ **Post date:** [December 12, 2018, 6:53am UTC](https://forum.ansible.com/t/high-memory-usage-when-running-multiple-ansible-playbooks-on-localhost/29244/1 "2018-12-12T06:53:50Z")
I have a requirement to run lots of playbooks parallely on localhost. When I run 100 playbooks parallely (a simple one that executes the a shell command - date) , I see that it consumes lot of resources (memory ~ 7GiB and CPU ~ 200%) Is this on expected lines ? here is what I am trying to run --------------------main.yml----------------------------------------- - hosts: 127.0.0.1 connection: local gather\_facts: no tasks: - name: Get date shell: date register: result - debug: var: result.stdout ---
**Author:** ![Karl\_Auer](https://avatars.discourse-cdn.com/v4/letter/k/cc9497/32.png) [@Karl\_Auer](https://forum.ansible.com/u/Karl_Auer)\ **Post date:** [December 12, 2018, 6:55am UTC](https://forum.ansible.com/t/high-memory-usage-when-running-multiple-ansible-playbooks-on-localhost/29244/2 "2018-12-12T06:55:40Z")
what happens if you leave Ansible out and just fire off 100 copies of date? ---
**Author:** ![pradeep.hk](https://avatars.discourse-cdn.com/v4/letter/p/8c91f0/32.png) [@pradeep.hk](https://forum.ansible.com/u/pradeep.hk)\ **Post date:** [December 12, 2018, 7:24am UTC](https://forum.ansible.com/t/high-memory-usage-when-running-multiple-ansible-playbooks-on-localhost/29244/3 "2018-12-12T07:24:53Z")
When I fire off 100 copies of date, I hardly see any change in memory usage. I even fired 100 copies of the following script, but I donot see any change in memory usage even when 100 such processes are running in parallel ---
**Author:** ![Karl\_Auer](https://avatars.discourse-cdn.com/v4/letter/k/cc9497/32.png) [@Karl\_Auer](https://forum.ansible.com/u/Karl_Auer)\ **Post date:** [December 12, 2018, 9:27am UTC](https://forum.ansible.com/t/high-memory-usage-when-running-multiple-ansible-playbooks-on-localhost/29244/4 "2018-12-12T09:27:23Z")
That script does not run 100 copies of date. It runs 100 copies of an empty loop which may well be optimised out of existence. I’m not saying that Ansible is not the problem, but you do need to start with a fair comparison. Also, your playbook is running 100 times on one system; you may find it is hardly a problem if you run those 100 scripts on 100 remote systems. Regards, K. ---
**Author:** ![Kai\_Stian\_Olstad](https://avatars.discourse-cdn.com/v4/letter/k/e19adc/32.png) [@Kai\_Stian\_Olstad](https://forum.ansible.com/u/Kai_Stian_Olstad)\ **Post date:** [December 12, 2018, 9:39am UTC](https://forum.ansible.com/t/high-memory-usage-when-running-multiple-ansible-playbooks-on-localhost/29244/5 "2018-12-12T09:39:10Z")
It sound about right. For small execution Ansible uses about 50MB of memmory, 50MB \* 100 is about 5GB. But since you are running the same playbook on the same host you could instead do this In inventory add [testgroup] testhost[001:100 And in ansible.cfg set fork=100 And the playbook - hosts: testgruop    connection: local    gather\_facts: no    tasks:      - name: Get date        shell: date        register: result      - debug:          var: result.stdout This will only use a little over 50MB and not GB of memory and a lot less CPU. Your test case seams to me not very relevant, but if it is, let Ansible fork and not the OS. ---
**Author:** ![pradeep.hk](https://avatars.discourse-cdn.com/v4/letter/p/8c91f0/32.png) [@pradeep.hk](https://forum.ansible.com/u/pradeep.hk)\ **Post date:** [December 12, 2018, 9:47am UTC](https://forum.ansible.com/t/high-memory-usage-when-running-multiple-ansible-playbooks-on-localhost/29244/6 "2018-12-12T09:47:19Z")
Few things: (1)As mentioned in my previous reply, I even ran the date command 100 times but no change in memory usage. In addition, I also tried a empty loop to keep the process running (2)It is my requirement to run the playbook on the localhost in response to a request to execute a job (there could be multiple such requests) ---
**Author:** ![pradeep.hk](https://avatars.discourse-cdn.com/v4/letter/p/8c91f0/32.png) [@pradeep.hk](https://forum.ansible.com/u/pradeep.hk)\ **Post date:** [December 12, 2018, 10:22am UTC](https://forum.ansible.com/t/high-memory-usage-when-running-multiple-ansible-playbooks-on-localhost/29244/7 "2018-12-12T10:22:14Z")
The requirement is to be able to launch a playbook on receiving a request (ie on demand). So, if 100 requests are received, it will result in 100 playbooks being executed parallely. You mentioned - Ansible uses about 50MB of memory per playbook. Is there something that can be done to optimize on that ? ---
**Author:** ![Kai\_Stian\_Olstad](https://avatars.discourse-cdn.com/v4/letter/k/e19adc/32.png) [@Kai\_Stian\_Olstad](https://forum.ansible.com/u/Kai_Stian_Olstad)\ **Post date:** [December 12, 2018, 11:32am UTC](https://forum.ansible.com/t/high-memory-usage-when-running-multiple-ansible-playbooks-on-localhost/29244/8 "2018-12-12T11:32:30Z")
50MB is just the minimum, it might be a lot more depending on playbook. I highly doubt there is much that can be done. Just a Hello World in python uses 7MB $ /usr/bin/time -f %M python -c 'print "Hello World"' Hello World Mem used in kB: 7316 An Ansible equivalent Hello World uses 48MB $ /usr/bin/time -f "Mem used in kB: %M" ansible-playbook test.yml PLAY [localhost] ---
**Author:** ![Kai\_Stian\_Olstad](https://avatars.discourse-cdn.com/v4/letter/k/e19adc/32.png) [@Kai\_Stian\_Olstad](https://forum.ansible.com/u/Kai_Stian_Olstad)\ **Post date:** [December 12, 2018, 3:56pm UTC](https://forum.ansible.com/t/high-memory-usage-when-running-multiple-ansible-playbooks-on-localhost/29244/9 "2018-12-12T15:56:16Z")
You don't even need to use dynamic inventory, this will do the same as your script but let ansible do the forking. #!/bin/bash ansible-playbook -i $(seq -s '' -f '%03g,' ${1-1}) -f ${1-1} main.yml And your your playbook must change to "hosts: all" ---
**Author:** ![pradeep.hk](https://avatars.discourse-cdn.com/v4/letter/p/8c91f0/32.png) [@pradeep.hk](https://forum.ansible.com/u/pradeep.hk)\ **Post date:** [December 13, 2018, 5:18am UTC](https://forum.ansible.com/t/high-memory-usage-when-running-multiple-ansible-playbooks-on-localhost/29244/10 "2018-12-13T05:18:56Z")
Got it. Thanks for your time. My requirement is to be able to sequence together tasks(to create a workflow) and after some prototyping, I found ansible playbook to meet that requirement. But then, scalability is turning out to be an issue. If we could daemonize the ansible process so that it can receive requests and execute playbook in a thread, that would scale. But then, that is not how ansible is supposed to work ---
**Author:** ![Jonathan\_Lozada\_De\_L](https://avatars.discourse-cdn.com/v4/letter/j/eb8c5e/32.png) [@Jonathan\_Lozada\_De\_L](https://forum.ansible.com/u/Jonathan_Lozada_De_L)\ **Post date:** [December 13, 2018, 11:45am UTC](https://forum.ansible.com/t/high-memory-usage-when-running-multiple-ansible-playbooks-on-localhost/29244/11 "2018-12-13T11:45:46Z")
tou ahould check out AWX