Ansible-test not loading collections from requirements.yml

Get Help
, ,
1

I’m attempting to run the integration tests for the ec2_vpc_nacl moudule of the amazon.aws collection and I’m running into an issue where the community.general collection is not installed in the container.

The command I’m running is:

ansible-test integration ec2_vpc_nacl --docker

(I’ve also tried using different container images such as fedora40).

The integration tests always fail because one of the tasks is attempting to use the community.general.json_query module, but the community.general collection isn’t being installed in the container by ansible-test.

TASK [ec2_vpc_nacl : Set helpful facts about subnets] **************************
fatal: [testhost]: FAILED! => {"msg": "template error while templating string: Could not load \"community.general.json_query\": 'Invalid plugin FQCN (community.general.json_query): unable to locate collection community.general'. String: {{ subnets | community.general.json_query('results[*].subnet.id') }}. Could not load \"community.general.json_query\": 'Invalid plugin FQCN (community.general.json_query): unable to locate collection community.general'"}

(NOTE: To get to the above task the fix from PR 2429 needs to be applied or use the --start-at-task "Fetch AZ availability")

I found that the community.general collection wasn’t added to the integration/tests/requirements.yml file so I added it locally to the file:

---
collections:
  - ansible.windows
  - ansible.utils # ipv6 filter
  - amazon.cloud # used by integration tests - rds_cluster_modify
  - community.general # used by integration test - ec2_vpc_nacl

However, even after adding the collection to the requirements.yml it is not found when running the test.

What do I need to do to get the collection installed in the container to be used during testing?

2

Not really an answer, but one of my personal bugbears: this should just be using standard Jinja, there’s nothing in there that requires the use of a non-standard filter.

        subnet_ids: "{{ subnets.results | map(attribute='subnet.id') }}"
        subnet_names: "{{ subnets.results | map(attribute='subnet.tags.Name') }}"

There are a limited set of circumstances where json_query is more usable than plain Jinja, and this is not one of them.

1 Like
3

You need to have it installed / checked out in the same directory structure from where you run ansible-test in amazon.aws. So if you are in /path/to/ansible_collections/amazon/aws, you need another directory /path/to/ansible_collections/community/general (with exactly the same /path/to) where community.general is installed (or its Git repo checked out). Only then will ansible-test find it (and automatically copy it into the Docker container).

The integration/tests/requirements.yml file is not used by ansible-test.

4

I have thought about changing the test to not use json_query as I think it would be cleaner too.

5

Thanks @felixfontein. I didn’t realize that and couldn’t find that documented anywhere.

6

Hi folks, I’ve just found this thread whilst I was raising my own issue somewhat related to this. I’m creating some unit tests for the community.beszel collection and slightly different to running integration tests I’m using the command: ansible-test units --docker --requirements

I noticed that the unit tests are failing in the container due to the ansible_collections.community.internal_test_tools not being found. I thought that ansible-test would use my requirements.yml file and install this collection dependency like it does with Python requirements.txt but that doesn’t seem to be the case based on @felixfontein’s last response.

To @branic point, I also couldn’t find:

So if you are in /path/to/ansible_collections/amazon/aws , you need another directory /path/to/ansible_collections/community/general (with exactly the same /path/to ) where community.general is installed (or its Git repo checked out). Only then will ansible-test find it (and automatically copy it into the Docker container).

Documented anywhere either. Where could this be documented, I don’t mind raising a PR for it.