Encountered problem while Updating Ansible Tower: "couldn't resolve module/action 'alternatives'"

Get Help
,
1

Hi,

I am really new here (and also to Ansible Tower), but our project has been using Ansible Tower a while, and they are now/currently in the process of upgrading the Tower, as follows:


Was: "1.2 AAP"??

Help==>About shows: Tower 3.8.3

Ansible 2.9.20 

ansible --version ==> ansible 2.9.20


To:


Now: "2.4 AAP"??

Help==>About shows" Ansible Automation Platform Controller 4.5.1 

ansible --version ==> ansible [core 2.15.9]

The upgrade installation has been done and we have been testing the new Tower by running some of our existing Templates, etc. to check what is working (and what is not) and we have encountered a problem where when we run one of our roles (a template), it is failing with the following messages (FYI, Verbosity was set to “1”):

Identity added: /runner/artifacts/92609/ssh_key_data (/runner/artifacts/92609/ssh_key_data)
Using /etc/ansible/ansible.cfg as config file
Vault password: 
ERROR! couldn't resolve module/action 'alternatives'. This often indicates a misspelling, missing collection, or incorrect module path.
The error appears to be in '/runner/project/roles/common/java1_8_role/tasks/main.yml': line 104, column 5, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
  ## Set default java to new version ##
  - name: Set default java to new version
    ^ here

That part of the main.yml looks like:

103  ## Set default java to new version ##
104  - name: Set default java to new version
105    alternatives:
106      link: /usr/bin/java
107      name: java
108      path: "{{ play_java_alternative }}" 
109      priority: 1
110    when: play_install_java

Also, we tried running a YML that also uses at least part of the same Set default java to new version and using ansible-playbook (command line), and it worked without error.

FYI:

  • Tower is on an RHEL8 machine
  • I think Tower is running under a user named “awx”
  • When we did the test using ansible-playbook, that command was run as a different user named “svc”.

As I said, I’m kind of new to Ansible, but I was wondering how we might go about trying to determine why that alternatives module is apparently not available when we use Tower, vs. being available when we run similar code using ansible-playbook as the “svc” user? Does Tower output any debug logging that would perhaps show up where it is looking for that module, etc.?

Thanks in advance, and if more detail is needed, please let me know and I will try to provide it!

Jim

2

Hello Jim,

Welcome to the wonderful world of Ansible :slight_smile:

TLDR:
There was some changes between ansible 2.9 and 2.15 (started happening at 2.10-ish). This might require a little research into what those changes will mean in your environment. The quick answer to what alternatives module is missing - look at this documentation community.general.alternatives module – Manages alternative programs for common commands — Ansible Community Documentation

This documentation goes over the change between 2.19 and 2.10 that should get you on the right path however as mentioned it may depend on your environment and existing playbooks.
https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_2.10.html

Good Luck and I hope this is helpful.

3

Hi jelly-is-bean,

Yes it is *SUPER * helpful, esp. for somone like me (and maybe us) that haven’t been “living” with Ansible for a number of years :wink: ! I’ve gone through the info in the links and I think that we are going to need to be modifying some code.

FYI specifically on that code snippet that I posted, we just did try to change the alternatives to community.general.alternatives but we still got the same error…

So not sure what they are going to do next :frowning: …

Jim

EDIT: FYI for the template/project, we are using SCM (pulls code from Bitbucket), and I was wondering - after we make the change in the playbook (where we changed alternative to community.general.alternative., do we need to bounce tower or something?

EDIT 2: We noticed with the new tower, when we run a template, there are 2 jobs that run. First one is a “Source Control” job that appears to be pulling source files from Bitbucket, etc. and second job is our template job. Also last night I had bumped up the Verbosity on our template to Debug… and I just noticed that for the last Source Control job in the Tower log we are seeing this:

TASK [Fetch galaxy collections from collections/requirements.(yml/yaml)] *******
[WARNING]: Unable to find
'/var/lib/awx/projects/_2312__gxstandarddlw/collections' in expected paths (use
-vvvvv to see paths)
skipping: [localhost]
TASK [Fetch galaxy roles and collections from requirements.(yml/yaml)] *********
skipping: [localhost]
PLAY RECAP *********************************************************************localhost                  : ok=3    changed=0    unreachable=0    failed=0    skipped=4    rescued=0    ignored=0

and when I do the “ls” command I get:

 ls -al /var/lib/awx/projects/_2312__gxstandarddlw/collections
ls: cannot access '/var/lib/awx/projects/_2312__gxstandarddlw/collections': No such file or directory

SO it seems like THAT may be the problem, that the source control job is failing to find the project :frowning: - but what could be causing THAT to happen?

4

I think that I just found the problem (but still don’t know why).

I turned up Verbosity on the Template to the maximum and re-ran our template and in the Tower logging it is showing:

ansible collection location = /runner/requirements_collections:/home/runner/.ansible/collections:/usr/share/ansible/collections

whereas the error we are seeing says:

TASK [Fetch galaxy collections from collections/requirements.(yml/yaml)] *******
[WARNING]: Unable to find
'/var/lib/awx/projects/_2312__gxstandarddlw/collections' in expected paths (use
-vvvvv to see paths)
skipping: [localhost]

Does anyone know why Tower would be looking at that /runner/requirements_collections:/home/runner/.ansible/collections:/usr/share/ansible/collections path for the collections?? Is that some kind default path, and if so, how should we set something to fix this problem?

Thanks!

EDIT : CORRECTION - I should have said:

Does anyone know why Tower would be looking at that `/var/lib/awx/projects` path for the collections??
5

Glad it was helpful to get you in the right direction … I am pretty new to using Ansible via an automation platform too (my favour is the upstream AWX project not the Red Hat Automation platform but they are similar).

The Red Hat documentation confirms that /var/lib/awx/projects folder is a default location however it can be configured to a different location

When I deployed AWX I created a PV/PVC (persistent volume and claim) for the projects and gave AWX permission to the location. Perhaps this is something similar to what you may need to do to get it working, ie create the location and/or give permissions. AAP will store its copy of the collection you reference in the playbook (the alternatives module as an example) and use it as part of its play.

6

Reviewing some of the Red Hat documentation, it points to a change between the old version and the current version you are running that you might want to review. Chapter 3. Migrating to automation execution environments Red Hat Ansible Automation Platform 2.4 | Red Hat Customer Portal

AAP now uses execution environments (containers that run the playbooks) and they can be created to contain all the relevant collections when running playbooks. As an example, an execution environment specific to playbooks for Microsoft Windows hosts would have the windows specific collections. Additionally, the chapter 6 (Converting playbooks) in that same link maybe helpful also.

I mention this as it may be a better position longterm to use them however this would be something you would need to review for your use case and environment.

7

Hi,

Can I ask a different question? Above you said:

When I deployed AWX I created a PV/PVC (persistent volume and claim) for the projects and gave AWX permission to the location. Perhaps this is something similar to what you may need to do to get it working, ie create the location and/or give permissions. AAP will store its copy of the collection you reference in the playbook (the alternatives module as an example) and use it as part of its play.

I was trying something today where I tried to zip up the collections directory that was under the /usr/share/ansible directory, which created a collections.zip file in the //usr/share/ansible directory, ok?

Then after that I changed to the /var/lib/awx/projects… my “plan” was to try to somehow get the collections directory over to the under the /var/lib/awx/projects directory… but after I did the cd command, I was surprised to find that there was suddenly/already a collections directory under the /var/lib/awx/projects directory :frowning: !!!

My guessing is that maybe Ansible or tower automatically unzipped that collections.zip file in the /usr directory and put the directory and files under the /var/lib/awx/projects directory???

Is that possible?

Thanks,
Jim

EDIT: I am asking the bove because if the above is what is happening and similar to what you said you had been doing, we think we can go this way short-term. I think we know we have to go the EE/execution environment route, but some of us feel that is going to take longer to implement (correctly).

8

Collections can be found in different locations (see reference 1):

  • /usr/share/ansible/collections
  • ~/.ansible/collections
  • within a project directory

As you are working with a platform that has undergone a significant upgrade it may be possible that Tower changed its default collections folder.

Regarding your question that Tower may have automatically unzipped your collections.zip file… I guess that might be possible (my apologises that is outside my experience) and perhaps there is someone in this community with more experience to answer that. However, my experience in general linux if there is a process that would do this, it would usually unzip to /tmp and then move it to a folder or unzip directly in that folder.

Another suggestion of what happened is pre-upgrade the collection was downloaded to the /usr/share location and post-upgrade the collection was downloaded to the projects folder. This wouldn’t account for the error you received during the playbook run which advised it was unable to find the location.

Have you reviewed the permissions on the two locations?

  • /usr/share/ansible/collections because this is where you found the collections stored
  • /var/lib/awx/project/... because this is where the collections are expected

Perhaps the error is not that it doesn’t exist but that it doesn’t have permissions and therefore it “doesn’t” exist for the platform.

Maybe you can configure Tower to use the previous location for the collections (see reference 1 but further down the article) as part of your short-term position.

[+] Reference 1: https://www.redhat.com/sysadmin/install-ansible-disconnected-node#:~:text=Deploy%20Ansible%20Collection%20on%20the%20control%20node