# update to fetch module, a great way to pull in selected log files and other things. **URL:** \ **Category:** Ansible Project\ **Created:** [April 19, 2012, 1:18am UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420 "2012-04-19T01:18:55Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [April 19, 2012, 1:18am UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/1 "2012-04-19T01:18:55Z")
The fetch module can be used to mirror a file structure on remote hosts locally. Ex: fetch src=/etc/motd dest=/srv/filetree This previously would create: /srv/filetree//motd I have fixed this on the integration branch to do: /srv/filetree//etc/motd Preserving the original directory structure. This enables relative shiny behavior like (in playbooks): - name: mirror important log files ($item) action: fetch src=$item dest=/srv/filetree with\_items: - /var/log/messages - /var/log/httpd/access.log - /var/log/httpd/error.log - /foo/database/whatever.log ETC. In the above example, if I had many database servers, I can view all of their logs at: /srv/filetree//foo/database/whatever.log for all values of HOSTNAME that the Ansible task is run against. –Michael ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [July 5, 2012, 12:27pm UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/2 "2012-07-05T12:27:56Z")
Request aside, tasks can have only one action in them, unless you are using with\_items. Playbooks are YAML, and each task is a hash. ---
**Author:** ![jpmens1](https://avatars.discourse-cdn.com/v4/letter/j/e9c0ed/32.png) [@jpmens1](https://forum.ansible.com/u/jpmens1)\ **Post date:** [July 5, 2012, 12:34pm UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/3 "2012-07-05T12:34:44Z")
>     - name: get the logs >       action: fetch src=/var/log/tomcat6/catalina.out dest=tomcat\_logs >       action: command /bin/chmod 0640 /var/log/yum.log >       action: fetch src=/var/log/yum.log dest=tomcat\_logs dest=tomcat\_logs                                            ^^^^ ^^^^ >       action: command /bin/chmod 0640 /var/log/messages >       action: fetch src=/var/log/messages dest=tomcat\_logs dest=tomcat\_logs                                             ^^^^ ^^^^ You have two (2) dest= parameters; only one is allowed. Use different dest= filenames to obtain all three files.         -JP ---
**Author:** ![Sriram\_Narayan](https://avatars.discourse-cdn.com/v4/letter/s/e68b1a/32.png) [@Sriram\_Narayan](https://forum.ansible.com/u/Sriram_Narayan)\ **Post date:** [July 6, 2012, 6:55am UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/4 "2012-07-06T06:55:37Z")
Thanks. Here is what I have now: - name: copy over the webapp action: copy src=$webapp\_path dest=/usr/share/tomcat6/webapps/$webapp\_name owner=sriram group=tomcat mode=0644 notify: - restart tomcat6 - chmod $logfile - get $logfile handlers: - name: restart tomcat6 action: service name=tomcat6 state=restarted - name: chmod $logfile action: command /bin/chmod 0640 $logfile with\_items: - /var/log/yum.log - /var/log/messages - name: get $logfile action: fetch src=$logfile dest=tomcat\_logs with\_items: - /var/log/tomcat6/catalina.out - /var/log/yum.log - /var/log/messages But it fails with: (and it returns a zero return code causing my CI tool to mark my deploy build green) ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [July 6, 2012, 10:41am UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/5 "2012-07-06T10:41:24Z")
We can't debug partial playbooks, Include the vars section please? Also, what ansible version? ---
**Author:** ![CWS](https://avatars.discourse-cdn.com/v4/letter/c/e495f1/32.png) [@CWS](https://forum.ansible.com/u/CWS)\ **Post date:** [July 6, 2012, 11:01am UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/6 "2012-07-06T11:01:58Z")
Actually, I think the problem is this part of the playbook:   - name: chmod $logfile     action: command /bin/chmod 0640 $logfile     with\_items:       - /var/log/yum.log       - /var/log/messages Shouldn't that be $item instead of $logfile? ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [July 6, 2012, 11:51am UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/7 "2012-07-06T11:51:52Z")
Yes. ---
**Author:** ![Sriram\_Narayan](https://avatars.discourse-cdn.com/v4/letter/s/e68b1a/32.png) [@Sriram\_Narayan](https://forum.ansible.com/u/Sriram_Narayan)\ **Post date:** [July 6, 2012, 2:09pm UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/8 "2012-07-06T14:09:27Z")
thanks. I didn’t realize it had to be named item only. However I now get ERROR: change handler (chmod $item) is not defined can’t these tasks be referenced under notify? Here is the full play: - name: deploy stock-ticker to tomcat hosts: stock\_ticker\_hosts vars\_files: - config.yml user: go sudo: True tasks: - name: ensure tomcat is installed action: yum pkg=tomcat6 state=installed - name: create go user and add go to group tomcat action: user name=go groups=tomcat,root - name: write the tomcat config file action: template src=tomcat-server-xml.j2 dest=/usr/share/tomcat6/conf/server.xml owner=go group=tomcat mode=0644 notify: - restart tomcat6 - name: copy over the webapp action: copy src=$webapp\_path dest=/usr/share/tomcat6/webapps/$webapp\_name owner=go group=tomcat mode=0644 notify: - restart tomcat6 - chmod $item - get $item handlers: - name: restart tomcat6 action: service name=tomcat6 state=restarted - name: chmod $item action: command /bin/chmod 0640 $item with\_items: - /var/log/yum.log - /var/log/messages - name: get $item action: fetch src=$item dest=tomcat\_logs with\_items: - /var/log/tomcat6/catalina.out - /var/log/yum.log - /var/log/messages ---
**Author:** ![CWS](https://avatars.discourse-cdn.com/v4/letter/c/e495f1/32.png) [@CWS](https://forum.ansible.com/u/CWS)\ **Post date:** [July 6, 2012, 2:11pm UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/9 "2012-07-06T14:11:59Z")
It appears to me that your $item is out of scope. ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [July 6, 2012, 2:15pm UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/10 "2012-07-06T14:15:13Z")
A few things: (A) You can’t use $item to define handler names. It probably should work though (patches accepted). Bonus points on being creative! The simplest workaround is to leave the fetch actions in the task section. (B) The chmod is unnecessary as you can pass the mode to copy or template calls directly. (See the module documentation for “file” and notes about the copy and template module). In fact, you’re doing this, so I’m not sure why the chmod notify is there. ---
**Author:** ![Sriram\_Narayan](https://avatars.discourse-cdn.com/v4/letter/s/e68b1a/32.png) [@Sriram\_Narayan](https://forum.ansible.com/u/Sriram_Narayan)\ **Post date:** [July 9, 2012, 7:39am UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/11 "2012-07-09T07:39:15Z")
Thanks. I was using chmod for files that I don’t create (logs). ---
**Author:** ![Dag\_Wieers](https://avatars.discourse-cdn.com/v4/letter/d/45deac/32.png) [@Dag\_Wieers](https://forum.ansible.com/u/Dag_Wieers)\ **Post date:** [July 9, 2012, 12:51pm UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/12 "2012-07-09T12:51:12Z")
This characteristics makes playbooks more verbose than they often need to be. I seem to be duplicating a lot of what the actions does in the name, just because I have to provide it. Most of the actions are self-explanatory, so the need to name each of them is cumbersome at best. From a 'usability' perspective, being able to group sets of actions (without requiring a name for each action) would be a big improvement. ---
**Author:** ![Dag\_Wieers](https://avatars.discourse-cdn.com/v4/letter/d/45deac/32.png) [@Dag\_Wieers](https://forum.ansible.com/u/Dag_Wieers)\ **Post date:** [July 9, 2012, 1:18pm UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/13 "2012-07-09T13:18:24Z")
Not setting the request aside, I would require the copy module to work on directories as well (or use globs). The use-case is that a directory with files need to be copied to the servers, and we don't want to list the files additionally in the playbook using with\_items. The files itself are checked-in into the repository together with the playbooks and templates. Would a patch implementing either:      action: copy src=files/etc/xinetd.d/ dest=/etc/xinetd.d/ (or perhaps:      action: copy src=files/etc/xinetd.d/ dest=/etc/ ) be accepted, or do we prefer the more versatile:      action: copy src=files/var/lib/tftpboot/maps/\*.ktl dest=/var/lib/tftpboot/maps/ Or both ? How about recursing through subdirectories ? ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [July 9, 2012, 1:38pm UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/14 "2012-07-09T13:38:43Z")
You are mistaken, you don’t have to provide a name. ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [July 9, 2012, 1:43pm UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/15 "2012-07-09T13:43:02Z")
> Not setting the request aside, I would require the copy module to work on > directories as well (or use globs). The use-case is that a directory with > files need to be copied to the servers, and we don’t want to list the > files additionally in the playbook using with\_items. I don’t see ansible implementing recursive cp, it doesn’t fit in well with the auth system to just invoke rsync (especially from paramiko or with sudo), and doing it any other way feels like reimplementing rsync. If you want this, you could transfer a tar ball and explode it, or use the git module to check out a subtree. ---
**Author:** ![jpmens1](https://avatars.discourse-cdn.com/v4/letter/j/e9c0ed/32.png) [@jpmens1](https://forum.ansible.com/u/jpmens1)\ **Post date:** [July 9, 2012, 1:59pm UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/16 "2012-07-09T13:59:48Z")
> Not setting the request aside, I would require the copy module to > work on directories as well (or use globs). If with\_items supported globs (or even reading from a file '\<', or pipe '|'), that would solve the problem as well, and enhance a lot of modules:   action: yum pkg=$item state=installed   with\_items:     - |/etc/printpackagelist ${fqdn}   action: copy src=$item dest=/tmp/dir/$item   with\_items:     - \< /etc/file.list Just dreaming... 😉         -JP ---
**Author:** ![Dag\_Wieers](https://avatars.discourse-cdn.com/v4/letter/d/45deac/32.png) [@Dag\_Wieers](https://forum.ansible.com/u/Dag_Wieers)\ **Post date:** [July 9, 2012, 2:17pm UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/17 "2012-07-09T14:17:17Z")
Right, none of the examples in the documentation reflect this. So I deduced the need for having name/action from those examples. ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [July 9, 2012, 2:33pm UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/18 "2012-07-09T14:33:49Z")
> > Not setting the request aside, I would require the copy module to > > work on directories as well (or use globs). > > If with\_items supported globs (or even reading from a file ‘\<’, or pipe > ‘|’), that would solve the problem as well, and enhance a lot of > modules: Just for everyone’s clarification, with\_items is a macro that creates lots of subtasks _PRIOR_ to communication with hosts, it can’t talk to remote hosts, which is why you can’t use vars\_files with it. Something like supporting an include would definitely be possible (patches accepted) with\_items: - include: /path/to/list.yml ---
**Author:** ![Dag\_Wieers](https://avatars.discourse-cdn.com/v4/letter/d/45deac/32.png) [@Dag\_Wieers](https://forum.ansible.com/u/Dag_Wieers)\ **Post date:** [July 9, 2012, 5:49pm UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/19 "2012-07-09T17:49:29Z")
Rereading the documentation, it specifically states that a name is required for every task:      "Every task must have a name, which is included in the output from       running the playbook." at: [http://ansible.github.com/playbooks.html#tasks-list](http://ansible.github.com/playbooks.html#tasks-list) It is likely that's where the confusion is coming from. ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [July 9, 2012, 5:58pm UTC](https://forum.ansible.com/t/update-to-fetch-module-a-great-way-to-pull-in-selected-log-files-and-other-things/11420/20 "2012-07-09T17:58:39Z")
> Rereading the documentation, it specifically states that a name is > required for every task: > > “Every task must have a name, which is included in the output from > running the playbook.” > > at: [http://ansible.github.com/playbooks.html#tasks-list](http://ansible.github.com/playbooks.html#tasks-list) Fix == s/must/should/