# using assemble with large files?

**URL:** <https://forum.ansible.com/t/using-assemble-with-large-files/12855>\
**Category:** Ansible Project\
**Created:** [May 23, 2013, 2:42am UTC](https://forum.ansible.com/t/using-assemble-with-large-files/12855 "2013-05-23T02:42:49Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Timothy\_Appnel](https://avatars.discourse-cdn.com/v4/letter/t/e5b9ba/32.png) [@Timothy\_Appnel](https://forum.ansible.com/u/Timothy_Appnel)\
**Post date:** [May 23, 2013, 2:42am UTC](https://forum.ansible.com/t/using-assemble-with-large-files/12855/1 "2013-05-23T02:42:49Z")

</div>

Was the assemble module designed to cap the size of the files it works with?

I was scripting something with Ansible today that had to grab a bunch of large files (logs) on remote servers and then concatenate them. I thought using the assemble module would be slicker than “shell cat \>\>” but on closer inspection the final file was truncated. Everything seemed to run fine. When I looked at the code I saw the assemble build the file in memory before writing it. Is limiting assemble to small files by design or is that something I should work up a patch to correct?

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\
**Post date:** [May 23, 2013, 3:10am UTC](https://forum.ansible.com/t/using-assemble-with-large-files/12855/2 "2013-05-23T03:10:35Z")

</div>

assemble is basically there to simulate conf.d file structures in applications that do not support conf.d, merely that.

I won’t say it’s by design though, so patches to make it awesome would be totally welcome.

---

<div class="post-metadata">

**Author:** ![Timothy\_Appnel](https://avatars.discourse-cdn.com/v4/letter/t/e5b9ba/32.png) [@Timothy\_Appnel](https://forum.ansible.com/u/Timothy_Appnel)\
**Post date:** [May 23, 2013, 9:37pm UTC](https://forum.ansible.com/t/using-assemble-with-large-files/12855/3 "2013-05-23T21:37:52Z")

</div>

[http://github.com/ansible/ansible/pull/3003](http://github.com/ansible/ansible/pull/3003)

I used this patch here to assemble a 9gb log file where before it died silently around 2gb.

Bonus: the code is smaller afterwards.

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\
**Post date:** [May 23, 2013, 9:40pm UTC](https://forum.ansible.com/t/using-assemble-with-large-files/12855/4 "2013-05-23T21:40:40Z")

</div>

Outstanding!

BTW, looks like you may have a tab on line 79.

---

<div class="post-metadata">

**Author:** ![Timothy\_Appnel](https://avatars.discourse-cdn.com/v4/letter/t/e5b9ba/32.png) [@Timothy\_Appnel](https://forum.ansible.com/u/Timothy_Appnel)\
**Post date:** [May 23, 2013, 10:21pm UTC](https://forum.ansible.com/t/using-assemble-with-large-files/12855/5 "2013-05-23T22:21:00Z")

</div>

Hum. Not picking up on the tab in my code here. The tmp.write line there is indented the correct amount.

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\
**Post date:** [May 23, 2013, 11:16pm UTC](https://forum.ansible.com/t/using-assemble-with-large-files/12855/6 "2013-05-23T23:16:20Z")

</div>

Might just be github formatting, I’ll check, thanks!