AWX upgrade failing to perform migration for upgrade of AWX operator 2.10.0 → 2.19.1.
Hello,
I’m in the process of trying to upgrade my AWX environment via the AWX operator. I’m going from version 2.10.0 of the operator to 2.19.1. However, it appears my upgrade is failing with errors for migrate_schema.yml:
awx-manager TASK [Create kubernetes job to perform the migration] ********************************
awx-manager fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed to retrieve requested object: b'{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"jobs.batch \\\\\"awx-migration-24.6.1\\\\\" is forbidden: User \\\\\"system:serviceaccount:awx:awx-operator-controller-manager\\\\\" cannot get resource \\\\\"jobs\\\\\" in API group \\\\\"batch\\\\\" in the namespace \\\\\"awx\\\\\"\",\"reason\":\"Forbidden\",\"details\":{\"name\":\"awx-migration-24.6.1\",\"group\":\"batch\",\"kind\":\"jobs\"},\"code\":403}\\n'", "reason": "Forbidden"}
awx-manager {"level":"error","ts":"2024-08-12T16:54:44Z","logger":"logging_event_handler","msg":"","name":"awx","namespace":"awx","gvk":"awx.ansible.com/v1beta1, Kind=AWX","event_type":"runner_on_failed","job":"746298583749467876","EventData.Task":"Create kubernetes job to perform the migration","EventData.TaskArgs":"","EventData.FailedTaskPath":"/opt/ansible/roles/installer/tasks/migrate_schema.yml:33","error":"[playbook task failed]","stacktrace":"github.com/operator-framework/ansible-operator-plugins/internal/ansible/events.loggingEventHandler.Handle\n\tansible-operator-plugins/internal/ansible/events/log_events.go:111"}
[...]
and
TASK [installer : Check for pending migrations] ********************************\r\ntask path: /opt/ansible/roles/installer/tasks/migrate_schema.yml:3\nok: [localhost] => {\"changed\": false, \"rc\": 0, \"return_code\": 0, \"stderr\": \"\", \"stderr_lines\": [], \"stdout\": \"15\\n\", \"stdout_lines\": [\"15\"]}\n\r\nTASK [installer : Get version of controller for tracking] **********************\r\ntask path: /opt/ansible/roles/installer/tasks/migrate_schema.yml:15\nok: [localhost] => {\"changed\": false, \"rc\": 0, \"return_code\": 0, \"stderr\": \"\", \"stderr_lines\": [], \"stdout\": \"24.6.1\\n\", \"stdout_lines\": [\"24.6.1\"]}\n\r\nTASK [installer : Sanitize instance version] ***********************************\r\ntask path: /opt/ansible/roles/installer/tasks/migrate_schema.yml:25\nok: [localhost] => {\"ansible_facts\": {\"version\": \"24.6.1\"}, \"changed\": false}\n\r\nTASK [installer : Create kubernetes job to perform the migration] **************\r\ntask path: /opt/ansible/roles/installer/tasks/migrate_schema.yml:33\nfatal: [localhost]: FAILED! => {\"changed\": false, \"msg\": \"Failed to retrieve requested object: b'{\\\"kind\\\":\\\"Status\\\",\\\"apiVersion\\\":\\\"v1\\\",\\\"metadata\\\":{},\\\"status\\\":\\\"Failure\\\",\\\"message\\\":\\\"jobs.batch \\\\\\\\\\\"awx-migration-24.6.1\\\\\\\\\\\" is forbidden: User \\\\\\\\\\\"system:serviceaccount:awx:awx-operator-controller-manager\\\\\\\\\\\" cannot get resource \\\\\\\\\\\"jobs\\\\\\\\\\\" in API group \\\\\\\\\\\"batch\\\\\\\\\\\" in the namespace \\\\\\\\\\\"awx\\\\\\\\\\\"\\\",\\\"reason\\\":\\\"Forbidden\\\",\\\"details\\\":{\\\"name\\\":\\\"awx-migration-24.6.1\\\",\\\"group\\\":\\\"batch\\\",\\\"kind\\\":\\\"jobs\\\"},\\\"code\\\":403}\\\\n'\", \"reason\": \"Forbidden\"}\n\r\nPLAY RECAP *********************************************************************\r\nlocalhost : ok=77 changed=0 unreachable=0 failed=1 skipped=65 rescued=0 ignored=0 \n","job":"3843762477658756621","name":"awx","namespace":"awx","error":"exit status 2"
Please let me know if any other info is needed. Thanks!