Accessing Previously Set Facts in a Module

Archives Ansible Developer
1

I apologize for asking this because I see a number of previous threads, but none of them hit exactly this point, or the answers don’t appear to have worked for me yet so I’m trying to figure out if I’m just completely missing something.

I have state that I need to persist between invocations of my module.

I’m following http://docs.ansible.com/developing_modules.html#module-provided-facts, but it appears to be pretty scarce (I know documentation is hard, so I’d love to contribute back the answer to the question I’m about to ask, since it’d probably fit there quite reasonably).

I have a list of strs which my module should read, possibly modify, and then store back.

The pattern for this, from various sequences of grepping around the example modules (which don’t seem to do this), reading the PlayBook and Runner source, and just jumping back and forth, seems like it should be something like:

…
facts = ansible_facts(module)
existing_services = facts.get(“services”, {})
new_services = do_stuff_with(services)
module.exit_json(…, ansible_facts={“services” : new_services}

But after the module runs once, nothing new appears in the facts (specifically, there’s no new “services” key, and no other new keys, and even assert "somethingIknowshouldbethere" in str(facts) fails so it doesn’t appear to get added to some subobject in there.).

Where should my keys be going?

https://github.com/ansible/ansible/blob/devel/lib/ansible/playbook/__init__.py#L519 makes it seem like the answer is “top-level”, but yeah, I don’t see them, so I must be missing something.

If my example isn’t concrete enough and in fact it looks correct, here’s my exact module: (preface: my slightly embarrassing internal comment is now public due to me pushing that to get it working – apologies, it’s been a slightly long day of frustration): https://github.com/Magnetic/nix-ansible/blob/3d37bceefe7b022ee0cd9ad31e6c3b773e6b4fe4/nix.py

As a tangent, maybe this function (get_all_facts) isn’t supposed to be public, I don’t see any documentation for it, and specifically about what kind of object it wants as an argument, but https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/facts.py#L2716-L2717 seems slightly strange, it seems to be expecting a module with a “filter” param specifically. Does that “# ======” section break mean “from here on down is private” or something, or why is it making that assumption? (I see that that code appears to be copy-pasted from the setup module, so maybe the answer is “it shouldn’t, and it was just copied over from there, which does have a filter parameter?”)

Appreciated,

-Julian

2

I apologize for asking this because I see a number of previous threads, but
none of them hit exactly this point, or the answers don't appear to have
worked for me yet so I'm trying to figure out if I'm just completely
missing something.

I have state that I need to persist between invocations of my module.

I'm
following http://docs.ansible.com/developing_modules.html#module-provided-facts,
but it appears to be pretty scarce (I know documentation is hard, so I'd
love to contribute back the answer to the question I'm about to ask, since
it'd probably fit there quite reasonably).

I have a list of strs which my module should read, possibly modify, and
then store back.

The pattern for this, from various sequences of grepping around the example
modules (which don't seem to do this), reading the PlayBook and Runner
source, and just jumping back and forth, seems like it should be something
like:

   ...
   facts = ansible_facts(module)

^ This will retrieve the base ansible facts, so unless you need to consume them, you wouldn't want to do this (also, you'd need `from ansible.module_utils.facts import *` for this to work as well)

   existing_services = facts.get("services", {})

^ This just returns an empty hash since services does not exist in the dict returned fro ansible_facts above.

   new_services = do_stuff_with(services)
   module.exit_json(..., ansible_facts={"services" : new_services}

^ When specifying a dictionary this way, the quotes should be left off of services. I suspect you are losing the dictionary you are passing back because of the json conversion that exit_json is doing. At this point, the ansible_facts variable should be set to a python dictionary.

But after the module runs once, nothing new appears in the facts
(specifically, there's no new "services" key, and no other new keys, and
even `assert "somethingIknowshouldbethere" in str(facts)` fails so it
doesn't appear to get added to some subobject in there.).

You will not see the services value from within your module, you are generating it within your module. The approach I took for returning facts and persisting data between runs was to use a combination of the module provided facts and local facts: http://docs.ansible.com/playbooks_variables.html#local-facts-facts-d

You can see how I'm doing it here: https://github.com/openshift/openshift-ansible/blob/master/roles/openshift_facts/library/openshift_facts.py but basically, I'm storing data that I want to persist in /etc/ansible/openshift.fact and then using that file to determine what "facts" to return based on the local facts, base defaults, and provider specified defaults.

Where should my keys be going?

https://github.com/ansible/ansible/blob/devel/lib/ansible/playbook/__init__.py#L519
makes it seem like the answer is "top-level", but yeah, I don't see them,
so I must be missing something.

If my example isn't concrete enough and in fact it looks correct, here's my
exact module: (preface: my slightly embarrassing internal comment is now
public due to me pushing that to get it working -- apologies, it's been a
slightly long day of frustration):
https://github.com/Magnetic/nix-ansible/blob/3d37bceefe7b022ee0cd9ad31e6c3b773e6b4fe4/nix.py

Changing line 71 to: 'ansible_facts={services: nix.services},' should be all that you need. You shouldn't need to import from ansible.module_utils.facts

Hope that helps,

Jason DeTiberus

3

Hey Jason, thanks so much for the info. I’m going to have a look at your Openshift module tomorrow, so the only thing in your reply that I’m not sure I understand initially is what you mean about services and line 71 – I’m passing a Python dictionary with one key, “services”, and expecting to retrieve that later, when the module runs again a second time – if I remove the quotes, I should get a NameError, because I have no local called services – am I misunderstanding what you mean, or is there more special sauce somewhere that’s doing something unexpected there?

4

The main thing is that any "facts" that you return in the ansible_facts dictionary is only going to be available after your module is run, it won't be persisted in any way by default. Your module will have to provide a way to persist the data between runs. In my case, I used the local_facts approach for storing the data so that it can also be retrieved through local_facts when ansible is run against the host even if my openshift_facts module has not been run.

Don't mind me on the syntax bit, I don't use a single language with enough regularity to keep all the syntax clear in my head, I was thinking of the way you would build a python dictionary using dict() instead of {}.

Hope that clarifies things a bit.

5

Hah understandable.

What specifically is the lifecycle of facts then if they’re not persisted? When you say “between runs” – what I’m looking for is for

ansible-playbook foo.yaml

where foo.yaml contains

  • name: first step
    mymodule: service=foo

  • name: second step
    mymodule: service=bar

and then goes off and runs some other task file as well, with service=baz,

that by the time I see service=baz, I should have a list of all three, “foo”, “bar”, and “baz” – that’s all the persistence I need, and I want it to disappear once ansible-playbook exits. Is even that not what is persisted with facts? If so what possibly can you use them for if they disappear as soon as the module exits? Might as well just use Python objects, which at least will persist until the process exits?

Thanks again,

-J

6

Okay, so the facts will be available to the rest of the playbook (and any included roles, playbooks, tasks, etc after the module has been called). I'm not sure how you would access the runtime facts from within the module, or if it is even possible. If you try to retrieve the facts using ansible.module_utils.facts, you will get the ansible generated facts only.

The main purpose of facts are to make decisions later within the playbook. The current modules that aggregate actions (yum, apt, etc) are currently done by workarounds in the ansible code and not natively in the module. I *believe* that there are plans for v2 to make this more generic, but someone from Ansible would have to confirm/deny that statement.

In my case, I need to persist the data between playbook runs, not just multiple module runs within a playbook. You might have to require a parameter to our module for the user to provide the output of hostvars[inventory_hostname] to consume the current facts as a workaround.

7

modules don't get data passed to them automatically, so if you add a
fact, you need to explicitly pas it to the module

about with_items optimization, this has been on our wishlist for a long time now

8

Ah cool, thanks for the confirmation. And I’d need to persist the fact myself right? Or how would I be able to pass it to the module, and whatever way that is, why does it need to involve ansible at all at that point? Might as well just pass Python objects around outside of ansible’s framework (and AnsibleModule)?

To me I’d have expected ansible to pass me a Python object that it passes to each module during invocation where I can hang state, and then it disappears at the end of the run. Is that the kind of thing that’s on the wishlist?

9

Hello Julian,

Did you ever come up with a solution to this? I’m wanting to do the exact same thing.

Thanks,
Hank

10

I haven’t yet, but I took a break to work on a few other things, so I’ll be coming back to it next week I hope.

Ansible doesn’t make persisting to a file trivial yet either from what I can tell, since you don’t get access to ansible_remote_tmp (or I haven’t found it yet), so I don’t yet see a way to be able to even store state in a file during a run in a way that gets cleaned up at the end.

But if / when I come up with a solution I’ll post it here.