# unarchive is slow to decompress very big tar.gz file **URL:** \ **Category:** Ansible Project\ **Created:** [February 28, 2018, 5:24am UTC](https://forum.ansible.com/t/unarchive-is-slow-to-decompress-very-big-tar-gz-file/26864 "2018-02-28T05:24:23Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Check\_Peck](https://avatars.discourse-cdn.com/v4/letter/c/f04885/32.png) [@Check\_Peck](https://forum.ansible.com/u/Check_Peck)\ **Post date:** [February 28, 2018, 5:24am UTC](https://forum.ansible.com/t/unarchive-is-slow-to-decompress-very-big-tar-gz-file/26864/1 "2018-02-28T05:24:23Z")
I have a below playbook which I am trying to run but it takes forever. Size of “test.tar.gz” is 100GB. > * * * > > - hosts: TEST\_BOX > serial: 1 > tasks: > - name: copy and untar latest tar.gz file > unarchive: src=test.tar.gz dest=/data/tasks/files/ I am using ansible 2.4.3.0. Any thoughts how can I make it faster? ---
**Author:** ![Check\_Peck](https://avatars.discourse-cdn.com/v4/letter/c/f04885/32.png) [@Check\_Peck](https://forum.ansible.com/u/Check_Peck)\ **Post date:** [March 1, 2018, 8:20pm UTC](https://forum.ansible.com/t/unarchive-is-slow-to-decompress-very-big-tar-gz-file/26864/2 "2018-03-01T20:20:44Z")
Does anyone have any thoughts on this? ---
**Author:** ![Kai\_Stian\_Olstad](https://avatars.discourse-cdn.com/v4/letter/k/e19adc/32.png) [@Kai\_Stian\_Olstad](https://forum.ansible.com/u/Kai_Stian_Olstad)\ **Post date:** [March 1, 2018, 8:25pm UTC](https://forum.ansible.com/t/unarchive-is-slow-to-decompress-very-big-tar-gz-file/26864/3 "2018-03-01T20:25:50Z")
> Does anyone have any thoughts on this? Maybe. > \> > \> I have a below playbook which I am trying to run but it takes forever. > \> Size of "test.tar.gz" is 100GB. What is taking forever, the copying or the untar-ing? > \> --- > \> - hosts: TEST\_BOX > \> serial: 1 > \> tasks: > \> - name: copy and untar latest tar.gz file > \> unarchive: src=test.tar.gz dest=/data/tasks/files/ > \> > \> > \> I am using ansible 2.4.3.0. Any thoughts how can I make it faster? It depends on where the bottle neck is. Your are copying from your client, if that has a slower connection you could put the file on another machine/server and get the file from there instead. ---
**Author:** ![Check\_Peck](https://avatars.discourse-cdn.com/v4/letter/c/f04885/32.png) [@Check\_Peck](https://forum.ansible.com/u/Check_Peck)\ **Post date:** [March 2, 2018, 6:01pm UTC](https://forum.ansible.com/t/unarchive-is-slow-to-decompress-very-big-tar-gz-file/26864/4 "2018-03-02T18:01:12Z")
Is there any way to avoid copying? Can we not just untar over the wire if there is any way? ---
**Author:** ![Kai\_Stian\_Olstad](https://avatars.discourse-cdn.com/v4/letter/k/e19adc/32.png) [@Kai\_Stian\_Olstad](https://forum.ansible.com/u/Kai_Stian_Olstad)\ **Post date:** [March 2, 2018, 6:18pm UTC](https://forum.ansible.com/t/unarchive-is-slow-to-decompress-very-big-tar-gz-file/26864/5 "2018-03-02T18:18:13Z")
Not with unarchive. If you put the file on a web server you could do this - shell: curl -s [https://some.url/test.tar.gz](https://some.url/test.tar.gz) | tar xzC /dest But this solution is not idempotent ---
**Author:** ![Check\_Peck](https://avatars.discourse-cdn.com/v4/letter/c/f04885/32.png) [@Check\_Peck](https://forum.ansible.com/u/Check_Peck)\ **Post date:** [March 2, 2018, 7:54pm UTC](https://forum.ansible.com/t/unarchive-is-slow-to-decompress-very-big-tar-gz-file/26864/6 "2018-03-02T19:54:15Z")
Ok got it. But if I try this way by using shell module: > * * * > > - hosts: TEST\_BOX > serial: 1 > tasks: > - name: copy and untar latest deals.tar.gz file > shell: “cd /data/tasks/files/; tar -xvzf test.tar.gz” It doesn’t work and I get an error like this: Am I doing anything wrong with my above tasks? ---
**Author:** ![Kai\_Stian\_Olstad](https://avatars.discourse-cdn.com/v4/letter/k/e19adc/32.png) [@Kai\_Stian\_Olstad](https://forum.ansible.com/u/Kai_Stian_Olstad)\ **Post date:** [March 2, 2018, 8:23pm UTC](https://forum.ansible.com/t/unarchive-is-slow-to-decompress-very-big-tar-gz-file/26864/7 "2018-03-02T20:23:43Z")
You need to transfer the file to the host first. So that's why I in my example used curl, it will get the file and then send them in memory to tar that untar the files and save them to disk. ---
**Author:** ![Yevgen\_Lasman](https://avatars.discourse-cdn.com/v4/letter/y/f08c70/32.png) [@Yevgen\_Lasman](https://forum.ansible.com/u/Yevgen_Lasman)\ **Post date:** [June 3, 2018, 1:47pm UTC](https://forum.ansible.com/t/unarchive-is-slow-to-decompress-very-big-tar-gz-file/26864/8 "2018-06-03T13:47:19Z")
It have the same problem and I noticed that for my .tar.bz2 archove the next command is called `/bin/gtar --list -C /data/sync --show-transformed-names --use-compress-program=pbzip2 --exclude=data/tmp -f /tmp/data-backup.tar.bz2` and it indeed takes twice more time presumably because of --list option. The playbook part is ` - name: Unpack data dump unarchive: remote\_src: yes src: /tmp/data-backup.tar.bz2 dest: “{{ destination }}/sync” extra\_opts: [“–use-compress-program=pbzip2”,“–exclude=data/tmp”] tags: - restore - data-restore ` and having --list in the comman line does not make any sense, since I didn’t call it! ---
**Author:** ![Yevgen\_Lasman](https://avatars.discourse-cdn.com/v4/letter/y/f08c70/32.png) [@Yevgen\_Lasman](https://forum.ansible.com/u/Yevgen_Lasman)\ **Post date:** [June 3, 2018, 1:55pm UTC](https://forum.ansible.com/t/unarchive-is-slow-to-decompress-very-big-tar-gz-file/26864/9 "2018-06-03T13:55:59Z")
And the next command it calls is! `/bin/gtar --diff -C /data/sync --show-transformed-names --use-compress-program=pbzip2 --exclude=data/tmp -f /tmp/data-backup.tar.bz2`