Module documentation: attribute(s) for idempotency and related behaviors

Right now if attributes are used in module documentation, it’s mostly the standard attributes defined in ansible/lib/ansible/plugins/doc_fragments/action_common_attributes.py at devel · ansible/ansible · GitHub, i.e. check_mode, diff_mode, platform, action_group, and in special cases a few more. I haven’t seen the attributes feature used for self-defined attributes yet (but maybe I missed them, hints are welcome).

In any case, I’ve been longer thinking about using them to document more aspects, especially related to idempotency. Most modules are supposed to be idempotent, i.e. when you run them twice in the row with the same arguments, there should be no changes. Some modules cannot do that, for example because they execute user-supplied commands without understanding them, like ansible.builtin.command, or because they operate with APIs that do not allow to determine whether a change is actually needed or not, and you cannot find it out after executing the corresponding action either. Sometimes this also depends on module options; for example force=true sometimes disables idempotency, especially if the module’s output is not fully deterministic (example: force regeneration of a private key). Figuring out whether a module is idempotent, resp. under which conditions it is idempotent can be quite a lot of work (if the module documentation is long), especially if the result is “it’s not documented”. So being able to see this information on a glance would be very helpful, since idempotency is one of the main pillars of how Ansible should work.

So I went ahead and introduced an attribute idempotent in some of the collections I maintain, and use it for all modules with support: none, support: partial, or support: full - the former two with details that describe why and/or under which conditions the module is (not) idempotent -, and sometimes support: N/A (again with details). Here’s a list of the PRs, links to the collection documentation on the Ansible devel docsite, and a link to the attributes of an example module from the collection:

While working on this, I was wondering about a very related question: which parameters of a module will be used for updates, and which are only used for creation / when an action is applied, and for which the behavior is configurable? This is another aspect that’s sometimes also summed up under idempotency: if you run a task twice, but in the second invocation modify a parameter, does or under which conditions does the module update the managed object accordingly? Again, the general consensus is that modules should update the objects the manage to match the provided parameters, but there are a lot of cases where that isn’t possible or potentially destructive (an object has to be destroyed and re-created, which can lead to data loss for volumes, partitions, private keys, …) and thus potentially also configurable (the module cannot say whether reformatting a disk is acceptable or not if someone adjusted a parameter of the task). And then there’s interaction with parameters set / modified from outside the module (say a property that’s not supported by the module). Some of these things are documented, some others are not and result in bug reports from users who assume that the module should behave differently. Sometimes users don’t read the documentation, but if they do it would be great if that information is easy to find.

Again, here I think attributes are really helpful. The big question here is: should this also be summed up under idempotent? Or should there be another - or more than one - new attributes?

7 Likes

@felixfontein great idea to add this attribute, thanks! I’ve just opened issues in a few repos to add this, new contributors are especially welcome (but anyone can help), even partial help:

On the question, I see you use the details: field in the PRs to describe the conditions when the idempotency is partial: it looks good to me as as a place to put details. I think if there were too many fields, it could be hard to read too, but as it is in the PR it looks imo good.

1 Like

I used the details field only to describe when the module is not idempotent. I did not use it to describe which values are updated (under which circumstances). That information is not there yet.

Ah, ok, maybe we could start with adding this information to details: in a couple of modules and put the links in this topic. Maybe once it’s there it’d be easier to see how it can be reorganized for better readability based on those specific cases?

I’m not sure whether mixing that information together is a good idea, since this also affects support. For many modules it will be easy, but for the tricky ones support will get less helpful and details will get pretty long.

I’m suggesting adding it for visibility: it’s often easier to solve a problem when you see it in a real specific example

Is this type of extension something core and SC need to review/approve of?

No, the attributes feature was designed so that anyone can define their own attributes. The only special case are some special attributes which have additional info (action_group with the extra required field membership, and platform with the extra required field platforms).

It’s definitely nice if not everyone starts adding random attributes though, but that the same ones are used everywhere (if they’re used). ansible-core provides a docs fragment that helps with this (ansible/lib/ansible/plugins/doc_fragments/action_common_attributes.py at devel · ansible/ansible · GitHub), but that doesn’t stop anyone (including community, or RH’s Ansible collections team) from standardizing and using more than these.

3 Likes

I love the idea. Is it worth having some kind of community managed registry of common attributes, that could potentially be promoted into core in the future? idempotency, for example, feels like something that would potentially be valuable information on a much broader scale, but only if we were consistent with it’s use. A recommended “common” description (potentially tuned by folks with the word-smithing skills of the docs team) would also be nice.

I can fully understand if ansible-core wasn’t ready for an influx of “unproven” metadata (yet), and the core team aren’t necessarily the “right” gatekeepers, but having a middle-ground between “free for all” and “ansible-core managed” might be nice.

2 Likes

This was designed exactly this way for the reasons felixfontain and tremble state.

On the issue of ‘idempotency’ I really think it is a big commitment, many people use that word too easily and don’t always match the user’s expectations. This is why you won’t see most core actions promise this, even if it is a goal we strive for.

1 Like