update to fetch module, a great way to pull in selected log files and other things.

Archives Ansible Project
1

The fetch module can be used to mirror a file structure on remote hosts locally.

Ex:

fetch src=/etc/motd dest=/srv/filetree

This previously would create:

/srv/filetree//motd

I have fixed this on the integration branch to do:

/srv/filetree//etc/motd

Preserving the original directory structure.

This enables relative shiny behavior like (in playbooks):

  • name: mirror important log files ($item)
    action: fetch src=$item dest=/srv/filetree
    with_items:
  • /var/log/messages
  • /var/log/httpd/access.log
  • /var/log/httpd/error.log
  • /foo/database/whatever.log

ETC.

In the above example, if I had many database servers, I can view all of their logs at:

/srv/filetree//foo/database/whatever.log

for all values of HOSTNAME that the Ansible task is run against.

–Michael

2

Request aside, tasks can have only one action in them, unless you are using with_items.

Playbooks are YAML, and each task is a hash.

3

    - name: get the logs
      action: fetch src=/var/log/tomcat6/catalina.out dest=tomcat_logs
      action: command /bin/chmod 0640 /var/log/yum.log
      action: fetch src=/var/log/yum.log dest=tomcat_logs dest=tomcat_logs

                                           ^^^^ ^^^^

      action: command /bin/chmod 0640 /var/log/messages
      action: fetch src=/var/log/messages dest=tomcat_logs dest=tomcat_logs

                                            ^^^^ ^^^^

You have two (2) dest= parameters; only one is allowed. Use different
dest= filenames to obtain all three files.

        -JP

4

Thanks. Here is what I have now:

  • name: copy over the webapp
    action: copy src=$webapp_path dest=/usr/share/tomcat6/webapps/$webapp_name owner=sriram group=tomcat mode=0644
    notify:
  • restart tomcat6
  • chmod $logfile
  • get $logfile
    handlers:
  • name: restart tomcat6
    action: service name=tomcat6 state=restarted
  • name: chmod $logfile
    action: command /bin/chmod 0640 $logfile
    with_items:
  • /var/log/yum.log
  • /var/log/messages
  • name: get $logfile
    action: fetch src=$logfile dest=tomcat_logs
    with_items:
  • /var/log/tomcat6/catalina.out
  • /var/log/yum.log
  • /var/log/messages

But it fails with: (and it returns a zero return code causing my CI tool to mark my deploy build green)

5

We can't debug partial playbooks, Include the vars section please?

Also, what ansible version?

6

Actually, I think the problem is this part of the playbook:

  - name: chmod $logfile
    action: command /bin/chmod 0640 $logfile
    with_items:
      - /var/log/yum.log
      - /var/log/messages

Shouldn't that be $item instead of $logfile?

7

Yes.

8

thanks. I didn’t realize it had to be named item only. However I now get

ERROR: change handler (chmod $item) is not defined

can’t these tasks be referenced under notify?

Here is the full play:

  • name: deploy stock-ticker to tomcat
    hosts: stock_ticker_hosts
    vars_files:
  • config.yml
    user: go
    sudo: True
    tasks:
  • name: ensure tomcat is installed
    action: yum pkg=tomcat6 state=installed
  • name: create go user and add go to group tomcat
    action: user name=go groups=tomcat,root
  • name: write the tomcat config file
    action: template src=tomcat-server-xml.j2 dest=/usr/share/tomcat6/conf/server.xml owner=go group=tomcat mode=0644
    notify:
  • restart tomcat6
  • name: copy over the webapp
    action: copy src=$webapp_path dest=/usr/share/tomcat6/webapps/$webapp_name owner=go group=tomcat mode=0644
    notify:
  • restart tomcat6
  • chmod $item
  • get $item
    handlers:
  • name: restart tomcat6
    action: service name=tomcat6 state=restarted
  • name: chmod $item
    action: command /bin/chmod 0640 $item
    with_items:
  • /var/log/yum.log
  • /var/log/messages
  • name: get $item
    action: fetch src=$item dest=tomcat_logs
    with_items:
  • /var/log/tomcat6/catalina.out
  • /var/log/yum.log
  • /var/log/messages
9

It appears to me that your $item is out of scope.

10

A few things:

(A)

You can’t use $item to define handler names. It probably should work though (patches accepted). Bonus points on being creative!

The simplest workaround is to leave the fetch actions in the task section.

(B)

The chmod is unnecessary as you can pass the mode to copy or template calls directly. (See the module documentation for “file” and notes
about the copy and template module). In fact, you’re doing this, so I’m not sure why the chmod notify is there.

11

Thanks. I was using chmod for files that I don’t create (logs).

12

This characteristics makes playbooks more verbose than they often need to be. I seem to be duplicating a lot of what the actions does in the name, just because I have to provide it.

Most of the actions are self-explanatory, so the need to name each of them is cumbersome at best. From a 'usability' perspective, being able to group sets of actions (without requiring a name for each action) would be a big improvement.

13

Not setting the request aside, I would require the copy module to work on directories as well (or use globs). The use-case is that a directory with files need to be copied to the servers, and we don't want to list the files additionally in the playbook using with_items.

The files itself are checked-in into the repository together with the playbooks and templates.

Would a patch implementing either:

     action: copy src=files/etc/xinetd.d/ dest=/etc/xinetd.d/

(or perhaps:

     action: copy src=files/etc/xinetd.d/ dest=/etc/

) be accepted, or do we prefer the more versatile:

     action: copy src=files/var/lib/tftpboot/maps/*.ktl dest=/var/lib/tftpboot/maps/

Or both ? How about recursing through subdirectories ?

14

You are mistaken, you don’t have to provide a name.

15

Not setting the request aside, I would require the copy module to work on
directories as well (or use globs). The use-case is that a directory with
files need to be copied to the servers, and we don’t want to list the
files additionally in the playbook using with_items.

I don’t see ansible implementing recursive cp, it doesn’t fit in well with the auth system to just invoke
rsync (especially from paramiko or with sudo), and doing it any other way feels like reimplementing rsync.

If you want this, you could transfer a tar ball and explode it, or use the git module to check out a subtree.

16

Not setting the request aside, I would require the copy module to
work on directories as well (or use globs).

If with_items supported globs (or even reading from a file '<', or pipe
'|'), that would solve the problem as well, and enhance a lot of
modules:

  action: yum pkg=$item state=installed
  with_items:
    - |/etc/printpackagelist ${fqdn}

  action: copy src=$item dest=/tmp/dir/$item
  with_items:
    - < /etc/file.list

Just dreaming... :wink:

        -JP

17

Right, none of the examples in the documentation reflect this. So I deduced the need for having name/action from those examples.

18

Not setting the request aside, I would require the copy module to
work on directories as well (or use globs).

If with_items supported globs (or even reading from a file ‘<’, or pipe
‘|’), that would solve the problem as well, and enhance a lot of
modules:

Just for everyone’s clarification,

with_items is a macro that creates lots of subtasks PRIOR to communication with hosts, it can’t talk to remote hosts, which is why you can’t
use vars_files with it.

Something like supporting an include would definitely be possible (patches accepted)

with_items:

  • include: /path/to/list.yml
19

Rereading the documentation, it specifically states that a name is required for every task:

     "Every task must have a name, which is included in the output from
      running the playbook."

at: http://ansible.github.com/playbooks.html#tasks-list

It is likely that's where the confusion is coming from.

20

Rereading the documentation, it specifically states that a name is
required for every task:

“Every task must have a name, which is included in the output from
running the playbook.”

at: http://ansible.github.com/playbooks.html#tasks-list

Fix == s/must/should/