Host lists, facts and handlers

Archives Ansible Project
1

Hi,

I do regularly have the problem that I want to manage directory
contents. To make things easier, we take an every day example such as
/etc/apt/sources.list.d on a Debian system. While I am aware that there
is a special apt module in ansible, I still chose to tackle this issue
"manually" because my ultimate goal is generic directory contents
management, not apt management. I chose apt as an example since that's
easier to reproduce for you.

I want:
  - files written in the directory by ansible
  - when a file ceases to be managed by ansible, the file should be
    removed. This is made easier by the fact that the ansible-managed
    files do match a certain glob, here: zda-*.
  - If a file doesn't change, it should not be touched. This includes
    that it is not desired that a file gets deleted and recreated with
    the same contents in the same ansible run.
  - Files placed into the directory manually by the local admin should
    remain untouched.

Here is my code:
==> ./site.yml <==

2

First ansible run:
[51/5036]mh@drop:~/git/zgansibletest (master * u+1) (crm) $ ansible-playbook --ask-become-pass --inventory=hosts.yml site.yml

<snip />

TASK [common : zda-sid-mc.list] *********************************************************************************************************************
changed: [sid01]
changed: [sid02]

RUNNING HANDLER [common : apt update] ***************************************************************************************************************
changed: [sid01]

Here is the problem, the task zda-sid-mc.list is changed on both host but only one of them run the handler.
Why this happen i don't know but at fist glance it looks like a bug.
What is your Ansible version?

You could also try more detailed logging to see if that gives more info:
  ANSIBLE_STDOUT_CALLBACK=debug ansible-playbook -vvv --ask-become-pass --inventory=hosts.yml site.yml

PLAY [clean up after apt configuration] *************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************************************
ok: [sid01]

Since the handler didn't run you only have one host in the second play.

The first host is fine, but the -stretch- file is not removed on the second host.

Second ansible run:
[52/5037]mh@drop:~/git/zgansibletest (master * u+1) (crm) $ ansible-playbook --ask-become-pass --inventory=hosts.yml site.yml

<snip />

TASK [common : zda-sid-mc.list] *********************************************************************************************************************
ok: [sid01]
ok: [sid02]
[WARNING]: Could not match supplied host pattern, ignoring: apt_action_hosts
PLAY [clean up after apt configuration] *************************************************************************************************************
skipping: no hosts matched

PLAY RECAP ******************************************************************************************************************************************
sid01 : ok=7 changed=0 unreachable=0 failed=0
sid02 : ok=7 changed=0 unreachable=0 failed=0

This is an obvious no-op, with ansible not doing anything on the second host,
but I would have expected it to remove the -stretch- file anyway

You have linked the removal of the files on the creating of zda-sid-mc.list, since this was created in you first run it's not created now, as you can see the ok status on zda-sid-mc.list task.
And because of that the handler will not be called so you second play doesn't run since there are no hosts in the group.

However, the way to get there is unsatisfactory. Can somebody explain
what is happening and tell me how to do things right? If inserting debug
code helps, I'll happily do this, but I do not yet know too well how to
do this, so please suggest pasteable things. Thanks in advance!

A mention above, it might be a bug in the notify/handler.

You could just run everything on all hosts all the time, and not create a runtime group that you later run a play against.
Because your solution now is not idempotent as you see on your second run when something fails in the play before.

3

After reading the documentation on add_host I don't think this is a but, it's just how add_host works.

The documentation says
"This module bypasses the play host loop and only runs once for all the hosts in the play, if you need it to iterate use a with_ directive."

So this is probably the case when used in handlers to, it only run once.

4

Hi Kai,

thanks for helping!

> First ansible run:
> [51/5036]mh@drop:~/git/zgansibletest (master * u+1) (crm) $ ansible-playbook --ask-become-pass --inventory=hosts.yml site.yml

<snip />

> TASK [common : zda-sid-mc.list] *********************************************************************************************************************
> changed: [sid01]
> changed: [sid02]
>
> RUNNING HANDLER [common : apt update] ***************************************************************************************************************
> changed: [sid01]

Here is the problem, the task zda-sid-mc.list is changed on both host but only one of them run the handler.
Why this happen i don't know but at fist glance it looks like a bug.
What is your Ansible version?

All systems in the test setup are Debian unstable, which ansible 2.5.0.

You could also try more detailed logging to see if that gives more info:
  ANSIBLE_STDOUT_CALLBACK=debug ansible-playbook -vvv --ask-become-pass --inventory=hosts.yml site.yml

That's 98 kB, too much for this list. I have uploadd to
http://q.bofh.de/~mh/stuff/ansible-log.txt

What confuses me is that no handling of the presentsourceslistfiles
variable is logged at all.

You have linked the removal of the files on the creating of zda-sid-mc.list, since this was created in you first run it's not created now, as you can see the ok status on zda-sid-mc.list task.

Have I? I don't think so. From the creation of zda-sid-mc.list, I only
notify the apt update handler which only does apt update. Building the
present/wantedsourceslistfiles is always done in the common task, and
always executed in the zzhandleapt task.

At least that's what I wanted to write. Didn't I do that?

Greetings
Marc

5

Hi Kai, again,

thanks for helping!

6

> First ansible run:
> [51/5036]mh@drop:~/git/zgansibletest (master * u+1) (crm) $ ansible-playbook --ask-become-pass --inventory=hosts.yml site.yml

<snip />

> TASK [common : zda-sid-mc.list] *********************************************************************************************************************
> changed: [sid01]
> changed: [sid02]
>
> RUNNING HANDLER [common : apt update] ***************************************************************************************************************
> changed: [sid01]

Here is the problem, the task zda-sid-mc.list is changed on both host but only one of them run the handler.
Why this happen i don't know but at fist glance it looks like a bug.
What is your Ansible version?

All systems in the test setup are Debian unstable, which ansible 2.5.0.

You could also try more detailed logging to see if that gives more info:
  ANSIBLE_STDOUT_CALLBACK=debug ansible-playbook -vvv --ask-become-pass --inventory=hosts.yml site.yml

That's 98 kB, too much for this list. I have uploadd to
http://q.bofh.de/~mh/stuff/ansible-log.txt

What confuses me is that no handling of the presentsourceslistfiles
variable is logged at all.

The tasks run, but you don't see the content of the variable, you would need to use debug for that.
And the log show it handles both notify but only one host is added by add_host.

You have linked the removal of the files on the creating of zda-sid-mc.list, since this was created in you first run it's not created now, as you can see the ok status on zda-sid-mc.list task.

Have I? I don't think so. From the creation of zda-sid-mc.list, I only
notify the apt update handler which only does apt update. Building the
present/wantedsourceslistfiles is always done in the common task, and
always executed in the zzhandleapt task.

At least that's what I wanted to write. Didn't I do that?

Your task "delete source.list.d files" is on role zzhandleapt so that will only be run/deleted when the notify is run, and notify is on zda-sid-mc.list task.

If the only go is to run apt update, why don't you just notify apt module directly?

Create the handler as so.

   - name: apt update
     apt:
       update_cache: yes

7

I would say don't use add_host it will just complicate thing for you.
You had a solution that work that you posted on this list back in November, why not just use that one? It didn't use add_host.

8

Hi Kai,

sorry for not getting back to you (and this list) any earlier; the
ansible stuff is just something I'm trying to grasp, and the paid-for
stuff kept prioritizing itself over and over again.

I would say don't use add_host it will just complicate thing for you.
You had a solution that work that you posted on this list back in November,
why not just use that one? It didn't use add_host.

The November version kept calling apt update over and over again for
every change it did, and while this is only time-consuming for apt[1],
it might be contraproductive in some other use cases of this construct.
Remember, I'm trying to build a standard idiom, and the apt case ist
just an easy example that anybody can easily try.

Greetings
Marc

[1] execution speed is currently my main beef with ansible, so I'd like
my code not be the fault for constant annoyance about things being slow.

9

Hi Kai,

(apology for taking six weeks to answer from other mail applies hier as
well)

> > > First ansible run:
> > > [51/5036]mh@drop:~/git/zgansibletest (master * u+1) (crm) $ ansible-playbook --ask-become-pass --inventory=hosts.yml site.yml
> >
> > <snip />
> >
> > > TASK [common : zda-sid-mc.list] *********************************************************************************************************************
> > > changed: [sid01]
> > > changed: [sid02]
> > >
> > > RUNNING HANDLER [common : apt update] ***************************************************************************************************************
> > > changed: [sid01]
> >
> > Here is the problem, the task zda-sid-mc.list is changed on both
> > host but only one of them run the handler.
> > Why this happen i don't know but at fist glance it looks like a bug.
> > What is your Ansible version?
>
> All systems in the test setup are Debian unstable, which ansible 2.5.0.
>
> > You could also try more detailed logging to see if that gives more
> > info:
> > ANSIBLE_STDOUT_CALLBACK=debug ansible-playbook -vvv
> > --ask-become-pass --inventory=hosts.yml site.yml
>
> That's 98 kB, too much for this list. I have uploadd to
> http://q.bofh.de/~mh/stuff/ansible-log.txt
>
> What confuses me is that no handling of the presentsourceslistfiles
> variable is logged at all.

The tasks run, but you don't see the content of the variable, you would need
to use debug for that.

So even with -vvv I won't see variables being handled without explicit
debug being written down?

And the log show it handles both notify but only one host is added by
add_host.

But why?

Your task "delete source.list.d files" is on role zzhandleapt so that will
only be run/deleted when the notify is run, and notify is on zda-sid-mc.list
task.

That notify is a leftover from the original try; the current version is
meant to built a list of hosts that need "apt update" in the
apt_action_hosts lists, which the zzhandleapt being called conditionally
for all hosts in the list from the site.yml.

If the only go is to run apt update, why don't you just notify apt module
directly?

Because that will run apt update multiple times per host, resulting in
wasted time, and not all jobs called this way are idempotent.

Greetings
Marc