There is a long-standing issue with policy clashes: AWX disallows environment variables starting with ANSIBLE_ for anything that isn’t coming from ansible-core, while it was decided project wide in 2019 that all environment variables defined by plugins or modules for configuration should use the prefix ANSIBLE_. This makes it impossible to use certain plugin configurations especially from community collections with AWX.
One proposal is to add an exception to not disallow environment variables starting with ANSIBLE_COMMUNITY_, and stick to these for community collections (and potentially other ones). This only works around this problem for community collections, but it’s at least a start
I would think that plugin environment variables, as a best practice, should use NAMESPACE_COLLECTION_ prefixes (COMMUNTIY_GENERAL_, COMMUNITY_NETWORK_, etc). Similar to how we treat role variables.
This still presents a presents a problem for the ansible.* collections, but then AWX could maintain credentials pertaining to ansible.* collections directly so that custom credentials should never need to be created for them (thus skipping this validation step).
Using ANSIBLE_COMMUNITY_ would be problematic since that is still prefixed with ANSIBLE_ and therefore AWX would have to account weirdly for exceptions.
I don’t think we should add a static allowlisted prefix like ANSIBLE_COMMUNITY_. As you said it doesn’t really solve for everything.
I’m also not in favor of NAMESPACE_COLLECTION_ prefixes (unless they are ANSIBLE_NAMESPACE_COLLECTION_ which doesn’t solve the issue), because they don’t have a common prefix; even worse, it makes sense for namespaces to match existing products or companies in some cases, which could have their own defined environment variables that end up with the same prefix.
I mentioned this in the original issue:
AWX may have a legitimate security concern, but they could help mitigate this issue by having an allow-list and/or block-list for specific variables that are allowed.
I would be in favor of AWX allowing its users to specifically opt-in to allowing certain variables or prefixes; that enabled administrators to make their own choices about what to override.
Using another prefix might also be a solution, like ANSIBLECONTENT_. Then you still have the common prefix ANSIBLE (without underscore), and everything starting with ANSIBLE_ is private to ansible-core, and everything starting with ANSIBLECONTENT_ is not.
fwiw, when we decided to use the ANSIBLE_ prefix it predated any real use of collections, and I’d argue that indeed in a collection based world it’s wrong for anything not part of ansible-core. Any plugin in an external collection at this point using ANSIBLE_ as a prefix should probably deprecate it.
I’m not a fan of a generic prefix.
I think I’d be ok with <NAMESPACE>_<NAME>_<PLUGIN_?>_<CONFIG_NAME> but I’m not convinced we really need to concern ourselves with any form of enforcement. But with some collections living in the ansible namespace, we still have potential problems.
Regardless, asking why AWX restricts the env vars seems like a question we should continue pulling at.
While, at the same time, you’re enforcing that the ANSIBLE_ prefix is only for ansible-core stuff? SCNR
In a collection based world and, additionally, where Ansible means the Ansible community package for a lot of people, I suggest that core should move to ANSIBLE_CORE_ prefix. At least in the long run.
And then suggest to collections to use <NAMESPACE>_<NAME>_$SOMETHING.
Of course, this would mean some work for the core team and would take some time to implement. But what do you think about this?
We should be clear that when we talk about the ANSIBLE_ prefix in this topic, it does not mean that it’s necessarily the entire prefix used in a collection. For example, all of my collection’s env vars use ANSIBLE_HASHI_VAULT_.
What’s at issue is that AWX cannot/does not have any way to distinguish these since it looks only for ANSIBLE_.
I don’t agree that it’s wrong for anything outside of core. The ANSIBLE_ prefix alone should be more about the general product Ansible and not core itself. To that end,
This isn’t such a bad idea, and would be a lot easier to implement than every collection having to change their stuff. AWX can then switch to restricting only ANSIBLE_CORE_ prefix which seems to be its intention.
Of course, there is no actual issue in core either, it’s really only AWX that has a problem, and we could sidestep anyone having to change any prefixes if AWX would allow its own users to selectively decide which vars/prefixes they want to allow to be overridden.
ansible-core is the real ansible. Unfortunately there were some politics that came into play and we ended up renaming the package, but in the end, ansible-core is the original and real ansible.
Well, and here we have a lot of disagreement. Ansible is also the name of the whole ecosystem, which includes a lot of other things that have Ansible in name: ansbile-lint, Ansible collections, Ansible roles, the Ansible community, Ansible navigator, Ansible Automation Platform, Ansible creator, etc. And these are just things whose name starts with Ansible, and thus have a similar reason to use environment variables starting with ANSIBLE_.
ansible-core might contain the real ansible CLI tool, use the ansible Python module name, etc., but I don’t think it has a special claim on the ANSIBLE_ prefix for environment variables.
ansible-config now has a validate option that checks all ANSIBLE_ environment variables and CAN include installed plugins. Sadly this doesn’t fix the older versions, but going forward is a way to check these dynamically vs using a static list.
Will the real Ansible, pls stand up! (SCNR …mostly just wanted to use that acronym cuz I haven’t seen it before and had to google it)
So ignoring all the technical/political hurdles - what is the impact on end users if we change any of the existing environment variables? So if ANSIBLE_HASHI_VAULT_ had to change to something like ANSIBLECONTENT_HASHI_VAULT…Is there some magic aliasing that happens for people using the older name or do they all have to change their environment variables when 'whatever change` happens?
Aside from ‘ansible’ having been overloaded to death, to the point I have to ask people ‘what do you mean by Ansible’ in almost every conversation. The ANSIBLE_ prefix has already been extensively used, changing it now and making it backwards compatible is a monumental task no matter what your ‘ansible scope’ is. I would argue that creating ‘better tools’ going forward (like ansible-config doing the validation, based on installed plugins) is a better solution than start changing standards.
Also, this is a good thing to point at before we create new standards, we should take our time to consider the ramifications, specially when adoption is extended beyond the intention. ANSIBLE_ for environment and ansilbe_ for internal vars were prefixes that were supposed to be exclusive to core, the move to collections changed that w/o anybody taking time to consider the implications of that change.
I wouldn’t be able to remove the old ones except during a major release. I would add the new ones and try to deprecate the old ones (or frankly, I might never remove them, because this change doesn’t seem right to me, and it only affects AWX users).
I already added vars entries which are also preceded with ansible_hashi_vault_ and AWX does not have this hard-coded restriction on ansible vars, only env vars.
I also hasn’t seen it before and just had to look it up
This is actually an interesting way to frame this. Sure, at one point in time, ANSIBLE_ was exclusive to the ansible binary/package, but now it is the inherited birthright of the collections born directly from ansible as part of Ansible’s legacy.
NAMESPACE_COLLECTION_ should be good practice, but ANSIBLE_NAMESPACE_COLLECTION_ should be just as legal, i.e. ANSIBLE_ shouldn’t be exclusive to ansible-core since it hasn’t been since 2.9~2.10.
The main problem is AWX has blacklisted ANSIBLE_, nearly 6 years ago before collections were introduced, under the guise that it is exclusive to the ansible binary and in turn a security concern. Regardless of the politics of it, the ANSIBLE_ prefix is not exclusive to core in practice (at least not in today’s ecosystem).
If there are still security concerns, then there needs to be a way to address these concerns. If AWX can use ansible-config to programmatically blacklist an array of known environment vars, that would alleviate the problem. If not programmatically, then at least a static list could be generated. Then AWX only needs to blacklist environment vars specific to core.
I’m sorry, I seemed to not have stated it correctly, but that was my point.
The move to collections also moved existing variables with those names into collections/3rd party/community and this was not something we considered at the time and how we arrived at the current state of having a ‘core convention’ expanded and adopted to every 3rd party plugin.
Instead of making a change of standards at this point (very costly, no matter which/where the change is made), i’m proposing to adopt the validation I’ve just added to ansible-config (which CAN consider 3rd party ansible plugins) as a solution.
Right now validation just issues warnings/errors, but I can easily add a --json option to give you a ‘processing friendly’ list.
But we already have a way to list all ‘usable’ variables by config from core and/or plugins in ansible-config, the new validation just points out ‘possible spelling errors’.
