Module writing guidelines

Archives Ansible Project
1

Hello guys,

I was wondering if there are any specific module writing guidelines? Some of the modules tend to print stack trace or the standard python error when it does not get the required commandline parameter. I think its a good idea to standardize the output of the modules when it encounters an error or even when it executes successfully. For example, when the params accepted by the module are not specified, it prints a JSON message telling what params were supplied and what params the module accepts. Similarly, for the output, I was hoping to get JSON output rather than see something like

localhost | success | rc=0 >> {
// module output
}

we could have something like

2

Hello guys,

I was wondering if there are any specific module writing guidelines?

http://ansible.github.com/moduledev.html

Some of the modules tend to print stack trace or the standard python error when it does not get the required commandline parameter. I think its a good idea to standardize the output of the modules when it encounters an error or even when it executes successfully. For example, when the params accepted by the module are not specified, it prints a JSON message telling what params were supplied and what params the module accepts. Similarly, for the output, I was hoping to get JSON output rather than see something like

We already have a standard for relaying user messages, it is “msg”.

If the module shares a traceback or stderr/stdout that is just a bonus and not required.

localhost | success | rc=0 >> {
// module output
}

we could have something like

[
{
host: ,
rc: ,
output:
]

What the CLI outputs for commands has no bearing on what the returns actually are.

Use the API and you will find out this is already there.

3

I checked a lot of modules, even the core modules and all of them don’t seem to have a particular pattern of standard. Some have the fail_json methods and some do not. I have already started sending in pull requests (user: nix85) to you by making changes to a few modules.

I think we can finalize on a few methods like fail_json and exit_json and allow modules to be added to the repo only if they contain these standard functions.

Let me know your thoughts Michael.

4

I think we can finalize on a few methods like fail_json and exit_json and allow modules to be added to the repo only if they contain these standard functions.

Yeah that was already the plan. Main thing is to not modify any behavior or input/output in doing so, and to test all of the corner cases to high degrees of coverage.

--Michael

5

Ok, here's the plan, since so many people seem to want this, and I *kind of* want this (common code), but have mostly been concerned about breaking people's modules in live situations and giving Python too many exemptions.

I'll come up with a list of functions and convert a module, and include something like a modules/stub, and once that is there we can
work on applying it to other modules for those that are interested, and then we can start cleaning stuff up. Basically this means a bit more utility functions not less, and yeah, I have preferences that I want to make sure are in there,
and code review is a lousy way for me to get someone else to do that for me (wouldn't be fair, -- move this over to the left, no no, more to the right, no to the left…). Once that's out, we can tweak them as needed.

I should have something this week.

I will also experiment with how I feel about ansible replacing a common block of code dynamically on transfer. I dislike this exemption to python/core modules behaving specially from a theoretical perspective, but would like to prevent
future confusion, so it might be an acceptable compromise provided hacking/test-module can still do the substitutions. Build time would confuse people, but if this just transparently worked it might be ok.

Yes, it makes the ansible modules kind of an internal API, so there will have to be a reasonably rigorous level of "thou shalt not change the return signature of this function, nor it's calling conventions, except to add keyword arguments
that are optional". We'll have to make that clear with a pretty evident skull and crossbones in the module.

Beyond all that, changing the modules is going to require we test them pretty thoroughly before release. So, if we do this, this probably slides our release a couple of weeks to make sure all the corners are well explored. By the very nature
of the modules, they require live testing, so not all of them are things we can unit test to 100% coverage on all platforms very easily. But I'm ok with that, and I think it will be worth it.

Basically a module if it wants the standard python code block will include like this:

{{ template_ansible_python_block_here }}

And it will be runtime, not build time.

--Michael

6

Sounds great Michael. Ill be happy to help with testing the modules and helping convert existing modules into the suggested new format.
At some point in time, we would have to do this. We definitely are in a better position now as we don’t have to rewrite too many modules.

Another question that I had is - what happens if people still don’t conform to the module writing guildelines? Is ansible going to throw an error or tell us that our module is not written in a particular way?

7

Yep, the time to do this is now.

We both have enough modules that it has become something worthwhile, and also that we don't have too many.

The classes we create here will be easy to use, and probably something like

module = AnsibleModule(arguments_hash_to_be_defined_later) # automatically does all the argument loading, parsing, and basic error handling, but more can be done later
module.exit_json(results)
module.fail_json(results)

To the question of rejecting modules, definitely not. Ansible treats all modules as scripts, and allows people to write modules in any language, so there will be no code to restrict or disable a module that does not use the import magic. However, it will be a requirement for a module to be included in the core so that is ships with Ansible, so new modules must look like the others.

I want people to be able to super-easily write modules in bash, Ruby, Perl, or whatever they like. Part of my reason for wanting to resist this was making it clear that modules *were* just scripts (but obviously, idempotent resource driven scripts). So, in order
to make it really easy, it is probably worthwhile to include, not in the core, but in examples/ at least an example of what a super simple Perl or Ruby hello world module would look like, so we could reference that in the docs.

And we can rewrite the module development docs when this goes out.

--Michael

8

Yep, the time to do this is now.

We both have enough modules that it has become something worthwhile, and also that we don’t have too many.

The classes we create here will be easy to use, and probably something like

module = AnsibleModule(arguments_hash_to_be_defined_later) # automatically does all the argument loading, parsing, and basic error handling, but more can be done later
module.exit_json(results)
module.fail_json(results)

Sounds a good way to start.

To the question of rejecting modules, definitely not. Ansible treats all modules as scripts, and allows people to write modules in any language, so there will be no code to restrict or disable a module that does not use the import magic. However, it will be a requirement for a module to be included in the core so that is ships with Ansible, so new modules must look like the others.

I want people to be able to super-easily write modules in bash, Ruby, Perl, or whatever they like. Part of my reason for wanting to resist this was making it clear that modules were just scripts (but obviously, idempotent resource driven scripts). So, in order
to make it really easy, it is probably worthwhile to include, not in the core, but in examples/ at least an example of what a super simple Perl or Ruby hello world module would look like, so we could reference that in the docs.

And we can rewrite the module development docs when this goes out.

I agree. Since ansible allows us to write modules in any language, we really cannot enforce any guidelines as such. It makes sense to look though for modules to follow these guidelines so that they can be included in the core.

9

Is it the time or place to talk about module tests? (e.g. Are all modules testable? Is there a base-level of testing we want to have for each core module?) I’m usually a big believer in TDD, but scratch my head when trying to apply it to configuration tools.

10

Not really interested in this. It’s too close to integration testing to be meaningful. Mock test code lies.