Deleting a collection with infra.ah_configuration does not work

Hello, thank you to who will know to point what I’m doing wrong with the ah_configuration collection.

I just can’ delete a specific AAH hosted community.general collection with infra.ah_configuration.collection role.
When I run the playbook / role, it connects to the AAH server, and then it’s just like if the collection I want to delete (community.general 6.6.6, for the beauty of the example) does not already exist (no change, no error, just a ok status, whereas I know the given collection does exist, as it appears in the interface where I can download it from).

The server is an automation hub 4.7.3 (aap2.4.2.3 10/18). The collection version is 2.0.4.

Yaml file with credentials :

---
ah_host: https://aah.domain.org/
ah_username: admin
ah_password: *****
ah_validate_certs: false

Yaml file describing collection to delete :

---
ah_collections:
  - namespace: community
    name: general
    version: 6.6.6
    state: absent

Last, the playbook itself:

---
- name: Efface une collection
  hosts: localhost
  gather_facts: no

  pre_tasks:
    - name: include vars
      ansible.builtin.include_vars:
        dir: ./vars
        extensions: ["yml"]

  roles:
    - infra.ah_configuration.collection

Playbook execution :

$ ansible-playbook collection_destroy.yml

PLAY [Efface une collection] ***************************************************************************************************************

TASK [include vars] ************************************************************************************************************************
ok: [localhost]

TASK [infra.ah_configuration.collection : Validating arguments against arg spec 'main' - An Ansible Role to update, or destroy Automation Hub Collections.] ***
ok: [localhost]

TASK [infra.ah_configuration.collection : Update or destroy Automation Hub Collection] *****************************************************
ok: [localhost] => (item={'namespace': 'community', 'name': 'general', 'version': '6.6.6', 'state': 'absent'})

TASK [infra.ah_configuration.collection : Sleep for 10 seconds and continue with play] *****************************************************
ok: [localhost]

TASK [infra.ah_configuration.collection : Update/Destroy collection | Wait for finish the update/destruction of collection] ****************
ok: [localhost] => (item={'failed': 0, 'started': 1, 'finished': 0, 'ansible_job_id': 'j875973451569.693423', 'results_file': '/root/.ansible_async/j875973451569.693423', 'changed': False, '__collection': {'namespace': 'community', 'name': 'general', 'version': '6.6.6', 'state': 'absent'}, 'ansible_loop_var': '__collection'})

PLAY RECAP *********************************************************************************************************************************
localhost                  : ok=5    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

I’m quite sure I do a stupid mistake somewhere or there’s a subtlety, but I just can not see what’s wrong.
thanks in advance, J

Hello @johan welcome to the community!

I just had the chance to replicate your issue. Here is what I tried:

If I set an incorrect namespace / collection name / version on the variables file, I get the same as you (just a changed=false, failed=false task result):

TASK [infra.ah_configuration.collection : Update/Destroy collection | Wait for finish the update/destruction of collection] ********************************
ok: [localhost] => (item={'failed': 0, 'started': 1, 'finished': 0, 'ansible_job_id': '441316101422.225237', 
'results_file': '/home/user/.ansible_async/441316101422.225237',
'changed': False, '__collection': {'namespace': 'community', 'name': 'general', 'version': '1.5.15', 'state': 'absent'}, 'ansible_loop_var': '__collection'})

PLAY RECAP *************************************************************************************************************************************************
localhost: ok=5    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

However, after setting correct var data I was able to delete the collection from my GalaxyNG 4.6.5 (notice the changed=true result):

TASK [infra.ah_configuration.collection : Update/Destroy collection | Wait for finish the update/destruction of collection] ********************************
changed: [localhost] => (item={'failed': 0, 'started': 1, 'finished': 0, 'ansible_job_id': '222931282161.225335',
'results_file': '/home/user/.ansible_async/222931282161.225335',
'changed': False, '__collection': {'namespace': 'cisco', 'name': 'cucm', 'version': '1.5.15', 'state': 'absent'}, 'ansible_loop_var': '__collection'})

PLAY RECAP *************************************************************************************************************************************************
localhost : ok=5    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Can you double check these vars to start with?

EDIT: Also, I tried to install on my GalaxyNG the 6.6.6 version of community.general to replicate the same environment as you with more precision, but I just wasn’t able (neither via the role or the UI). That’s the error I get when trying to install:

This smells fishy to me… Maybe it’s the collection itself the culprit here? You may try to uninstall it manually also, see if that old collection structure is affecting in some way…

Cheers!

Hello jbericat, thank you very much for yout input.

On my AAH, the community.general.6.6.6 I try to delete was obtained by synchronization with Ansible Galaxy (ie I configure the community repository by providing a requirements.yml file which includes every community.general collection version).
Indeed, my code behaves like if community.general.6.6.6 didn’t exist (had the same result when giving weird version number in the playbook), although this collection is actually present on my AAH, I can even manually download the tarball.

I wonder if there’s not an untold limitation, such as : the delete with infra.ah_configuration collection is only possible for user published collection, collections synchronized from Ansible Galaxy website have to be handled with appropriate requirements.yml file (in which case i would have to report other interesting behaviour too :slight_smile: ). This is just speculation from me.

In your example, I think you can’t manually upload community general 6.6.6 because, for some not very clearly documented reasons, AAH / Galaxy NG requirements for packaging regularly change.

I’m currently travelling now, but i will try the collection deletion role with user uploaded collection in a few days to see what it give,

I’ve been recently playin more and more with infra collections, they are very interesting, but from time to time there’s quite a shift between the readme file and the behaviour (default values, actually requisite variables, …).

thank you very much,
Johan

1 Like

No problem @johan I’m learning a few things on the way also :slight_smile:

With all the info you did provide on your last answer I can investigate further. Also, if we find-out something unusual about the collection we could “summon” its developer and ask for his opinion. Anyway; just out of curiosity I will try to dig further into the matter this weekend to see if I can get to any conclusions also.

Cheers and have a nice trip!

I think you messaged wrong user?

uops… sorry! and thanks for the heads-up

Hello,
for the record, I had the role working with a colldction from the published repository (had to increase the ah_timeout value to 30, though).

I didn’t investigate more yet, but it seems the behaviour of the role is to delete by default objects from the published repository, as seen on this log from the pulpcore-api service :

pulp [0f8e59736e7540019ac9c340f4f19cc0]: django.request:WARNING: Not Found: /api/galaxy/v3/plugin/ansible/content/published/collections/index/community/general/versions/6.6.6/

I will check in the collection code if I can find clues on how to change the api path used, so far the code doesn’t make much sense to me :slight_smile:

best regards, Johan

Hello @johan sorry for the delay,

Right now I’m trying out another approach in regards of your problem, but I’m having a few difficulties to make it work. I’ll keep trying during the next few hours though.

More precisely, what I noticed is that community collections synced from remote repositories (your scenario) doesn’t get added to the “Published” repository, but the “Community” one:

Sooo… what I’m trying now is to use the infra.ah_configuration.collection_remote role to remove the collection from the Community repo.

I’ll get back to you shortly, just wanted to let you know because you’ve been waiting since yesterday for an answer.

Cheers!

Got it!!

I’m sharing with you the whole playbook, it works like a “source of truth” (the collections specified on the requirements are the ones that are kept on the AH - that is, if you remove a collection from the requirements, it gets deleted from the AH instance:

---
- name: Updating Automation remote collection repository (Community)
  hosts: localhost
  connection: local
  gather_facts: false
  vars:
  # Define following vars here, or in ah_configs/ah_auth.yml
    ah_host: https://galaxy.lab.lan
    ah_username: admin
    ah_password: "XXXXXXXXXXX"
    ah_request_timeout: 30
    ah_path_prefix: galaxy
    ah_collection_remotes:
      - name: community
        url: https://galaxy.ansible.com/api/
        requirements:
          - name: community.general
            version: "6.6.7"
        token: "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
    ah_validate_certs: false
  roles:
    - infra.ah_configuration.collection_remote
...

On the flip side, this is not the ideal solution since the remote collection needs to be synced manually afterwards:

Notice that I just replaced the 6.6.6 version of the collection with the 6.6.7:

Now I’m trying to add the infra.ah_configuration.repository_sync role to do the whole thing at once. I’ll keep you posted :slight_smile:

Here, I believe this one wraps-it-all:

---
- name: Updating Automation remote collection repository (Community)
  hosts: localhost
  connection: local
  gather_facts: false
  vars:
  # Define following vars here, or in ah_configs/ah_auth.yml
    ah_host: https://galaxy.lab.lan
    ah_username: admin
    ah_password: "xxxxxxxxxxxxxxxxx"
    ah_request_timeout: 30
    ah_path_prefix: galaxy
    ah_collection_remotes:
      - name: community
        url: https://galaxy.ansible.com/api/
        requirements:
          - name: community.general
            version: "8.0.1"
        token: "xxxxxxxxxxxxxxxxxxxxxxxxxxx"
    ah_validate_certs: false
    ah_repositories:
      - name: community
  tags:
    - always
  roles:
    - infra.ah_configuration.collection_remote
    - infra.ah_configuration.repository_sync
...

Output:

(ansible_ve) [15:34:42] jbericat@localhost:~/DEV/repos/personal/ansible-forum$ ansible-playbook test_forum_johan_collection_remote.yml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
[DEPRECATION WARNING]: infra.ah_configuration.ah_repository_sync has been deprecated. The endpoint has been removed and is not supported in AAP 2.4 onwards. This feature will be removed from 
infra.ah_configuration in version 3.0.0. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

PLAY [Updating Automation remote collection repository (Community)] ***********************************************************************************************************************************************

TASK [infra.ah_configuration.collection_remote : Validating arguments against arg spec 'main' - An Ansible Role to create collection remotes in Automation Hub.] **************************************************
ok: [localhost]

TASK [infra.ah_configuration.collection_remote : Add Automation Hub Collection Remote repository] *****************************************************************************************************************
ok: [localhost] => (item={'name': 'community', 'url': 'https://galaxy.ansible.com/api/', 'requirements': [{'name': 'community.general', 'version': '8.0.1'}], 'token': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'})

TASK [infra.ah_configuration.collection_remote : Create Repository | Wait for finish the repository creation] *****************************************************************************************************
FAILED - RETRYING: [localhost]: Create Repository | Wait for finish the repository creation (50 retries left).
FAILED - RETRYING: [localhost]: Create Repository | Wait for finish the repository creation (49 retries left).
FAILED - RETRYING: [localhost]: Create Repository | Wait for finish the repository creation (48 retries left).
FAILED - RETRYING: [localhost]: Create Repository | Wait for finish the repository creation (47 retries left).
FAILED - RETRYING: [localhost]: Create Repository | Wait for finish the repository creation (46 retries left).
changed: [localhost] => (item={'failed': 0, 'started': 1, 'finished': 0, 'ansible_job_id': 'j908446099689.14269', 'results_file': '/home/jbericat/.ansible_async/j908446099689.14269', 'changed': False, '__collection_remote_item': {'name': 'community', 'url': 'https://galaxy.ansible.com/api/', 'requirements': [{'name': 'community.general', 'version': '8.0.1'}], 'token': 'xxxxxxxxxxxxxxxxxxxxxxxxxxx'}, 'ansible_loop_var': '__collection_remote_item'})

TASK [infra.ah_configuration.repository_sync : Validating arguments against arg spec 'main' - An Ansible Role to sync repositories in Automation Hub.] ************************************************************
ok: [localhost]

TASK [infra.ah_configuration.repository_sync : Sync Automation Hub repository] ************************************************************************************************************************************
ok: [localhost] => (item={'name': 'community'})

TASK [infra.ah_configuration.repository_sync : Repository Sync | Wait for finish the repository_sync to finish] ***************************************************************************************************
FAILED - RETRYING: [localhost]: Repository Sync | Wait for finish the repository_sync to finish (50 retries left).
FAILED - RETRYING: [localhost]: Repository Sync | Wait for finish the repository_sync to finish (49 retries left).
FAILED - RETRYING: [localhost]: Repository Sync | Wait for finish the repository_sync to finish (48 retries left).
FAILED - RETRYING: [localhost]: Repository Sync | Wait for finish the repository_sync to finish (47 retries left).
FAILED - RETRYING: [localhost]: Repository Sync | Wait for finish the repository_sync to finish (46 retries left).
FAILED - RETRYING: [localhost]: Repository Sync | Wait for finish the repository_sync to finish (45 retries left).
FAILED - RETRYING: [localhost]: Repository Sync | Wait for finish the repository_sync to finish (44 retries left).
FAILED - RETRYING: [localhost]: Repository Sync | Wait for finish the repository_sync to finish (43 retries left).
FAILED - RETRYING: [localhost]: Repository Sync | Wait for finish the repository_sync to finish (42 retries left).
FAILED - RETRYING: [localhost]: Repository Sync | Wait for finish the repository_sync to finish (41 retries left).
changed: [localhost] => (item={'failed': 0, 'started': 1, 'finished': 0, 'ansible_job_id': 'j272073159057.14504', 'results_file': '/home/jbericat/.ansible_async/j272073159057.14504', 'changed': False, '__repository_item': {'name': 'community'}, 'ansible_loop_var': '__repository_item'})
[WARNING]: This role 'repository_sync' and module 'ah_repository_sync' will be removed when support for AAP 2.3 ends in May of 2024. The module and role collection_repository_sync replaced it.

PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost                  : ok=6    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0  

I hope that works for you :wink:

EDIT:

I just noticed that the collection infra.ah_configuration.repository_sync will be replaced by infra.ah_configuration.collection_repository_sync on May’24. Still haven’t tried it yet, I’ll update the playbook above accordingly once done.

Also, be aware that I had to reboot my Galaxy instance twice while I was testing the infra.ah_configuration.repository_sync (it just got unresponsive via API). I wonder if that’s why the role is being replaced by the newer one…

1 Like

hello Jordi, thank you very much for this workaround which by the way helps me to anticipate my casc journey by showing how remote repository and synchronization roles work :slight_smile: I currently have too some issues with my AAH ( Synchronizing content from Galaxy fails with 403 Forbidden on Private Ansible Automation Hub 4.7.x - Red Hat Customer Portal and When syncing collections in the private Automation Hub, how to disable download all the dependencies? - Red Hat Customer Portal ), and I will test the playbook as soon as i solve them !

1 Like

Hello @johan,

I was reviewing my bookmarked posts and stumbled upon this one.

Did you have the chance to test the playbook already? If you need further assistance, feel free to ask. On the other hand, if you believe the information provided on this thread helped you find a solution, could you kindly tick the solved checkbox :white_check_mark: on the post you believe helped the most, please? That will set the thread as solved on the Get Help category, so others with the same problem may benefit from it .

Cheers

Hello Jordi, thank you very much for your follow up, I confirm the playbook works fine :slight_smile:
best regards,
Johan

1 Like

Hello @johan, glad to see that it worked for you :wink:

See you around!

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.