# Memory error when downloading 3.4GB file **URL:** \ **Category:** Ansible Project\ **Created:** [March 28, 2019, 11:55pm UTC](https://forum.ansible.com/t/memory-error-when-downloading-3-4gb-file/30016 "2019-03-28T23:55:40Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Sindhuja\_Koneru](https://avatars.discourse-cdn.com/v4/letter/s/ac91a4/32.png) [@Sindhuja\_Koneru](https://forum.ansible.com/u/Sindhuja_Koneru)\ **Post date:** [March 28, 2019, 11:55pm UTC](https://forum.ansible.com/t/memory-error-when-downloading-3-4gb-file/30016/1 "2019-03-28T23:55:40Z")
Hello, Has anyone come across this Memory error? It occurs when running backup.yml script which is to backup DB and SECRET KEY. Script is completely successful, until it reaches to a point when it needs to download a common.tar.gz file which is 3.4GB --Fails with an unpredictable memory error. Any suggestions or help please? ./setup.sh -b -i inventoryfile -e backup\_dir=/tower/bkpsetup TASK [backup : Download the common backup.] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* task path: /TowerSetup/tower\_binaries/ansible-tower-setup-bundle-3.4.2-1.el7/roles/backup/tasks/download.yml:31 \ ESTABLISH SSH CONNECTION FOR USER: root \ SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ‘IdentityFile=“/export/home/zsjkvh0/.ssh/id\_rsaDev”’ -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/e37aeee5bc abc.corp…com ‘/bin/sh -c ‘"’“‘echo ~root && sleep 0’”’"‘’ \ (0, ‘/root\n’, ‘’) \ ESTABLISH SSH CONNECTION FOR USER: root \ SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ‘IdentityFile=“/export/home/zsjkvh0/.ssh/id\_rsaDev”’ -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/e37aeee5bc abc.corp…com ‘/bin/sh -c ‘"’"’( umask 77 && mkdir -p “`echo /root/.ansible/tmp/ansible-tmp-1553783711.94-90505061625143`” && echo ansible-tmp-1553783711.94-90505061625143=“`echo /root/.ansible/tmp/ansible-tmp-1553783711.94-90505061625143`” ) && sleep 0’“'”‘’ skipping: [abc2.corp…com] =\> { “changed”: false, “skip\_reason”: “Conditional result was False” } skipping: [abc3.corp…com] =\> { “changed”: false, “skip\_reason”: “Conditional result was False” } \ (0, ‘ansible-tmp-1553783711.94-90505061625143=/root/.ansible/tmp/ansible-tmp-1553783711.94-90505061625143\n’, ‘’) Using module file /usr/lib/python2.7/site-packages/ansible/modules/net\_tools/basics/slurp.py \ PUT /root/.ansible/tmp/ansible-local-19291WC3\_1C/tmprxDde7 TO /root/.ansible/tmp/ansible-tmp-1553783711.94-90505061625143/slurp.py \ SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o ‘IdentityFile=“/export/home/zsjkvh0/.ssh/id\_rsaDev”’ -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/e37aeee5bc ‘[abc.corp…com]’ \ (0, ‘sftp\> put /root/.ansible/tmp/ansible-local-19291WC3\_1C/tmprxDde7 /root/.ansible/tmp/ansible-tmp-1553783711.94-90505061625143/slurp.py\n’, ‘’) \ ESTABLISH SSH CONNECTION FOR USER: root \ SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ‘IdentityFile=“/export/home/zsjkvh0/.ssh/id\_rsaDev”’ -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/e37aeee5bc abc.corp…com ‘/bin/sh -c ‘"’“‘chmod u+x /root/.ansible/tmp/ansible-tmp-1553783711.94-90505061625143/ /root/.ansible/tmp/ansible-tmp-1553783711.94-90505061625143/slurp.py && sleep 0’”’"‘’ \ (0, ‘’, ‘’) \ ESTABLISH SSH CONNECTION FOR USER: root \ SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ‘IdentityFile=“/export/home/zsjkvh0/.ssh/id\_rsaDev”’ -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/e37aeee5bc -tt abc.corp…com ‘/bin/sh -c ‘"’"’/usr/bin/python /root/.ansible/tmp/ansible-tmp-1553783711.94-90505061625143/slurp.py && sleep 0’“'”‘’ \ ESTABLISH SSH CONNECTION FOR USER: root \ SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ‘IdentityFile=“/export/home/zsjkvh0/.ssh/id\_rsaDev”’ -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/e37aeee5bc abc.corp…com ‘/bin/sh -c ‘"’“‘rm -f -r /root/.ansible/tmp/ansible-tmp-1553783711.94-90505061625143/ \> /dev/null 2\>&1 && sleep 0’”’"‘’ \ (0, ‘’, ‘’) The full traceback is: Traceback (most recent call last): File “/usr/lib/python2.7/site-packages/ansible/executor/task\_executor.py”, line 138, in run res = self.\_execute() File “/usr/lib/python2.7/site-packages/ansible/executor/task\_executor.py”, line 576, in \_execute result = self.\_handler.run(task\_vars=variables) File “/usr/lib/python2.7/site-packages/ansible/plugins/action/fetch.py”, line 95, in run slurpres = self.\_execute\_module(module\_name=‘slurp’, module\_args=dict(src=source), task\_vars=task\_vars) File “/usr/lib/python2.7/site-packages/ansible/plugins/action/ **init**.py”, line 831, in \_execute\_module res = self.\_low\_level\_execute\_command(cmd, sudoable=sudoable, in\_data=in\_data) File “/usr/lib/python2.7/site-packages/ansible/plugins/action/ **init**.py”, line 938, in \_low\_level\_execute\_command rc, stdout, stderr = self.\_connection.exec\_command(cmd, in\_data=in\_data, sudoable=sudoable) File “/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py”, line 1007, in exec\_command (returncode, stdout, stderr) = self.\_run(cmd, in\_data, sudoable=sudoable) File “/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py”, line 276, in wrapped display.vvv(return\_tuple, host=self.host) File “/usr/lib/python2.7/site-packages/ansible/utils/display.py”, line 183, in vvv return self.verbose(msg, host=host, caplevel=2) File “/usr/lib/python2.7/site-packages/ansible/utils/display.py”, line 206, in verbose self.display(“\<%s\> %s” % (host, msg), color=C.COLOR\_VERBOSE, screen\_only=True) MemoryError fatal: [abc.corp…com]: FAILED! =\> { “msg”: “Unexpected failure during module execution.”, “stdout”: “” } ---
**Author:** ![Stephen\_John\_Smoogen](https://avatars.discourse-cdn.com/v4/letter/s/ad7895/32.png) [@Stephen\_John\_Smoogen](https://forum.ansible.com/u/Stephen_John_Smoogen)\ **Post date:** [March 28, 2019, 11:58pm UTC](https://forum.ansible.com/t/memory-error-when-downloading-3-4gb-file/30016/2 "2019-03-28T23:58:16Z")
What architectures are you using? 3.4 gb is a common failure size for 32 bit processors ---
**Author:** ![Sindhuja\_Koneru](https://avatars.discourse-cdn.com/v4/letter/s/ac91a4/32.png) [@Sindhuja\_Koneru](https://forum.ansible.com/u/Sindhuja_Koneru)\ **Post date:** [March 29, 2019, 12:13am UTC](https://forum.ansible.com/t/memory-error-when-downloading-3-4gb-file/30016/3 "2019-03-29T00:13:00Z")
It’s Linux system with 64 bit (x86\_64) Running on 2 cpu x 8 memory, there are no other processes running except ansible. Im running the setup.sh as root user. I can see enough disk space and available memory, no clue why it throws memory error? Is it trying to load the entire file to some cache and fails? ---
**Author:** ![Sebastian\_Meyer](https://avatars.discourse-cdn.com/v4/letter/s/779978/32.png) [@Sebastian\_Meyer](https://forum.ansible.com/u/Sebastian_Meyer)\ **Post date:** [March 29, 2019, 12:14am UTC](https://forum.ansible.com/t/memory-error-when-downloading-3-4gb-file/30016/4 "2019-03-29T00:14:56Z")
Hi, ---
**Author:** ![Jonathan\_Lozada\_De\_L](https://avatars.discourse-cdn.com/v4/letter/j/eb8c5e/32.png) [@Jonathan\_Lozada\_De\_L](https://forum.ansible.com/u/Jonathan_Lozada_De_L)\ **Post date:** [March 29, 2019, 12:16am UTC](https://forum.ansible.com/t/memory-error-when-downloading-3-4gb-file/30016/5 "2019-03-29T00:16:28Z")
I think you might not have enough space to do this. You should contact red hat support and get them to help you. ---
**Author:** ![Sindhuja\_Koneru](https://avatars.discourse-cdn.com/v4/letter/s/ac91a4/32.png) [@Sindhuja\_Koneru](https://forum.ansible.com/u/Sindhuja_Koneru)\ **Post date:** [March 29, 2019, 12:26am UTC](https://forum.ansible.com/t/memory-error-when-downloading-3-4gb-file/30016/6 "2019-03-29T00:26:15Z")
Thanks for the reply Sebastian. It(backup.yml) is a inbuilt module provided in the ansible bundle. It is also running fetch.py during the download File “/usr/lib/python2.7/site-packages/ansible/plugins/action/fetch.py”, line 95, in run slurpres = self.\_execute\_module(module\_name=‘slurp’, module\_args=dict(src=source), task\_vars=task\_vars) ---
**Author:** ![Sindhuja\_Koneru](https://avatars.discourse-cdn.com/v4/letter/s/ac91a4/32.png) [@Sindhuja\_Koneru](https://forum.ansible.com/u/Sindhuja_Koneru)\ **Post date:** [March 29, 2019, 12:47am UTC](https://forum.ansible.com/t/memory-error-when-downloading-3-4gb-file/30016/7 "2019-03-29T00:47:45Z")
To download, ansible is using fetch module. fetch: dest: ’ ’ fail\_on\_missing: yes flat: yes src: ’ ’ ---
**Author:** ![Sindhuja\_Koneru](https://avatars.discourse-cdn.com/v4/letter/s/ac91a4/32.png) [@Sindhuja\_Koneru](https://forum.ansible.com/u/Sindhuja_Koneru)\ **Post date:** [March 29, 2019, 3:53am UTC](https://forum.ansible.com/t/memory-error-when-downloading-3-4gb-file/30016/8 "2019-03-29T03:53:26Z")
I found that it’s a big in the Ansible module. Module fetch cannot download larger files , instead we need to try synchronize module with mode set to pull and it is working. ---
**Author:** ![system](https://us1.discourse-cdn.com/ansible/original/1X/1882d9a01da5d8996a862359b0913cc568a39b84.svg) [@system](https://forum.ansible.com/u/system)\ **Post date:** [March 29, 2019, 4:25am UTC](https://forum.ansible.com/t/memory-error-when-downloading-3-4gb-file/30016/9 "2019-03-29T04:25:41Z")
fetch will automatically use slurp if become is true, fetch itself should not have issues with large files, but slurp uses RAM \> x2 the file size due to copying the full file into memory and then base64 encoding it. ---
**Author:** ![Dominic\_D\_Apice](https://avatars.discourse-cdn.com/v4/letter/d/a698b9/32.png) [@Dominic\_D\_Apice](https://forum.ansible.com/u/Dominic_D_Apice)\ **Post date:** [June 7, 2019, 10:00pm UTC](https://forum.ansible.com/t/memory-error-when-downloading-3-4gb-file/30016/10 "2019-06-07T22:00:23Z")
[https://access.redhat.com/articles/3344101](https://access.redhat.com/articles/3344101) [https://docs.ansible.com/ansible/2.5/modules/fetch\_module.html](https://docs.ansible.com/ansible/2.5/modules/fetch_module.html)