Terraform backend credential missing following upgrade of AWX operator to 2.19.1

Hello,

We are in the process of upgrading our AWX / AWX Operator deployment.
Our AWX operator version is 2.11.0 (AWX version 23.7.0) and we are upgrading to version 2.19.1 (AWX version 24.6.1).
As part of the upgrade we have upgraded our external database from Postgres 13 to Postgres 15.

The upgrade seems to complete successfully and we are able to run jobs as normal in the upgraded deployment.

However we noticed that the Terraform backend configuration credential, which should be available in this version of AWX, is not present.

Any ideas or suggestions on why this credential type is not present?

Some db queries and logs are below but please let me know if any further details are needed.

$ sudo -u postgres psql -c "SELECT version();"
                                                 version
----------------------------------------------------------------------------------------------------------
 PostgreSQL 15.8 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 11.4.1 20231218 (Red Hat 11.4.1-3), 64-bit
(1 row)

$ sudo -u postgres psql -d awx_dev1 -c "select * from main_credentialtype where name ilike '%terraform%'"
could not change directory to "/root": Permission denied
 id | created | modified | description | name | kind | managed | inputs | injectors | created_by_id | modified_by_id | namespace
----+---------+----------+-------------+------+------+---------+--------+-----------+---------------+----------------+-----------
(0 rows)


Looking at the logs of the awx-operator-controller-manager, the migration part of the upgrade installer seems to have worked ok - see the log snippet below.

2024-11-07T13:37:07.966485615Z TASK [installer : Create kubernetes job to perform the migration] **************
2024-11-07T13:37:07.966499292Z task path: /opt/ansible/roles/installer/tasks/migrate_schema.yml:33
2024-11-07T13:37:07.966570280Z 
-------------------------------------------------------------------------------
2024-11-07T13:37:09.731507236Z {"level":"info","ts":"2024-11-07T13:37:09Z","logger":"proxy","msg":"Read object from cache","resource":{"IsResourceRequest":true,"Path":"/apis/batch/v1/namespaces/awx-dev1/jobs/awx-dev1-migration-24.6.1","Verb":"get","APIPrefix":"apis","APIGroup":"batch","APIVersion":"v1","Namespace":"awx-dev1","Resource":"jobs","Subresource":"","Name":"awx-dev1-migration-24.6.1","Parts":["jobs","awx-dev1-migration-24.6.1"]}}
2024-11-07T13:37:09.734272726Z {"level":"info","ts":"2024-11-07T13:37:09Z","logger":"proxy","msg":"Read object from cache","resource":{"IsResourceRequest":true,"Path":"/apis/batch/v1/namespaces/awx-dev1/jobs/awx-dev1-migration-24.6.1","Verb":"get","APIPrefix":"apis","APIGroup":"batch","APIVersion":"v1","Namespace":"awx-dev1","Resource":"jobs","Subresource":"","Name":"awx-dev1-migration-24.6.1","Parts":["jobs","awx-dev1-migration-24.6.1"]}}
2024-11-07T13:37:09.896860502Z 
2024-11-07T13:37:09.896902551Z --------------------------- Ansible Task StdOut -------------------------------
2024-11-07T13:37:09.896908772Z 
TASK [installer : Watch for the migration job to finish] ***********************
2024-11-07T13:37:09.896920795Z task path: /opt/ansible/roles/installer/tasks/migrate_schema.yml:42
2024-11-07T13:37:09.896925233Z 
2024-11-07T13:37:09.896929628Z -------------------------------------------------------------------------------
2024-11-07T13:37:09.896902301Z {"level":"info","ts":"2024-11-07T13:37:09Z","logger":"logging_event_handler","msg":"[playbook task start]","name":"awx-dev1","namespace":"awx-dev1","gvk":"awx.ansible.com/v1beta1, Kind=AWX","event_type":"playbook_on_task_start","job":"2399185321392484729","EventData.Name":"installer : Watch for the migration job to finish"}
2024-11-07T13:37:10.948047780Z {"level":"info","ts":"2024-11-07T13:37:10Z","logger":"proxy","msg":"Read object from cache","resource":{"IsResourceRequest":true,"Path":"/apis/batch/v1/namespaces/awx-dev1/jobs/awx-dev1-migration-24.6.1","Verb":"get","APIPrefix":"apis","APIGroup":"batch","APIVersion":"v1","Namespace":"awx-dev1","Resource":"jobs","Subresource":"","Name":"awx-dev1-migration-24.6.1","Parts":["jobs","awx-dev1-migration-24.6.1"]}}

<snipped>

024-11-07T13:37:39.467276751Z ----- Ansible Task Status Event StdOut (awx.ansible.com/v1beta1, Kind=AWX, awx-dev1/awx-dev1) -----
2024-11-07T13:37:39.467291769Z 
2024-11-07T13:37:39.467345565Z 
2024-11-07T13:37:39.467378145Z PLAY RECAP *********************************************************************
2024-11-07T13:37:39.467411823Z localhost                  : ok=92   changed=0    unreachable=0    failed=0    skipped=83   rescued=0    ignored=1   
1 Like

Iā€™m still encountering this issue and would greatly appreciate any suggestions or guidance. Thanks in advance for your help!