[DAB RBAC] Migration problem from 24.2.1 to 24.3.1 (or above)

Get Help
, ,
1

Hi,

We are currently trying to upgrade from 24.2.1 to 24.6.1 but we encoutered a blocking problem during the database migration.
(Our database is an external PostgreSQL 12).

The migration job is looping and the following stacktrace can be observed :

Operations to perform:
  Apply all migrations: auth, conf, contenttypes, dab_rbac, dab_resource_registry, main, oauth2_provider, sessions, sites, social_django, sso
Running migrations:
2024-12-05 07:26:14,338 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition NotificationTemplate Admin pk=RoleDefinition object (361) with 3 permissions
2024-12-05 07:26:14,362 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization NotificationTemplate Admin pk=RoleDefinition object (362) with 5 permissions
2024-12-05 07:26:14,381 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Inventory Admin pk=RoleDefinition object (363) with 6 permissions
2024-12-05 07:26:14,405 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Inventory Admin pk=RoleDefinition object (364) with 8 permissions
2024-12-05 07:26:14,424 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Inventory Use pk=RoleDefinition object (365) with 2 permissions
2024-12-05 07:26:14,437 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Inventory Adhoc pk=RoleDefinition object (366) with 2 permissions
2024-12-05 07:26:14,446 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Inventory Update pk=RoleDefinition object (367) with 2 permissions
2024-12-05 07:26:14,457 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition JobTemplate Admin pk=RoleDefinition object (368) with 4 permissions
2024-12-05 07:26:14,479 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization JobTemplate Admin pk=RoleDefinition object (369) with 5 permissions
2024-12-05 07:26:14,503 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition JobTemplate Execute pk=RoleDefinition object (370) with 2 permissions
2024-12-05 07:26:14,523 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition InstanceGroup Admin pk=RoleDefinition object (371) with 4 permissions
2024-12-05 07:26:14,552 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition InstanceGroup Use pk=RoleDefinition object (372) with 2 permissions
2024-12-05 07:26:14,580 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Project Admin pk=RoleDefinition object (373) with 5 permissions
2024-12-05 07:26:14,608 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Project Admin pk=RoleDefinition object (374) with 7 permissions
2024-12-05 07:26:14,632 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Project Update pk=RoleDefinition object (375) with 2 permissions
2024-12-05 07:26:14,659 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Project Use pk=RoleDefinition object (376) with 2 permissions
2024-12-05 07:26:14,684 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Team Admin pk=RoleDefinition object (377) with 4 permissions
2024-12-05 07:26:14,707 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Team Member pk=RoleDefinition object (378) with 2 permissions
2024-12-05 07:26:14,731 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Credential Admin pk=RoleDefinition object (379) with 4 permissions
2024-12-05 07:26:14,752 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Credential Admin pk=RoleDefinition object (380) with 6 permissions
2024-12-05 07:26:14,773 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Credential Use pk=RoleDefinition object (381) with 2 permissions
2024-12-05 07:26:14,780 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition ExecutionEnvironment Admin pk=RoleDefinition object (382) with 3 permissions
2024-12-05 07:26:14,787 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization ExecutionEnvironment Admin pk=RoleDefinition object (383) with 5 permissions
2024-12-05 07:26:14,794 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition WorkflowJobTemplate Admin pk=RoleDefinition object (384) with 5 permissions
2024-12-05 07:26:14,801 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization WorkflowJobTemplate Admin pk=RoleDefinition object (385) with 7 permissions
2024-12-05 07:26:14,808 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition WorkflowJobTemplate Execute pk=RoleDefinition object (386) with 2 permissions
2024-12-05 07:26:14,817 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition WorkflowJobTemplate Approve pk=RoleDefinition object (387) with 2 permissions
2024-12-05 07:26:14,827 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Member pk=RoleDefinition object (388) with 2 permissions
2024-12-05 07:26:14,834 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Admin pk=RoleDefinition object (389) with 46 permissions
2024-12-05 07:26:14,844 INFO     [-] awx.main.migrations._dab_rbac Created RoleDefinition Organization Audit pk=RoleDefinition object (390) with 10 permissions
Traceback (most recent call last):
[ ... ]
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/migrations/_dab_rbac.py", line 196, in migrate_to_new_rbac
    object = object_cls.objects.get(pk=role.object_id)  # WORKAROUND, role.content_object does not work in migrations
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/django/db/models/query.py", line 637, in get
    raise self.model.DoesNotExist(
__fake__.JobTemplate.DoesNotExist: JobTemplate matching query does not exist.

An old thread seemed related (with the same error message “JobTemplate matching query does not exist”) but it was something else.
Source: Need Help with awx 9.3.0 migration to 13.0.0

A migration to a closer version, 24.4.0 and then 23.4.1 has been tested but unfortunately with the same outcome.

The changelog is pretty clear on the 24.3.0 : the DAB RBAC was implemented but had a major problem which was fixed in 24.3.1.
And regarding the error message, it is clearly linked :

  File "/var/lib/awx/venv/awx/lib64/python3.11/site-packages/awx/main/migrations/_dab_rbac.py", line 196, in migrate_to_new_rbac
    object = object_cls.objects.get(pk=role.object_id)  # WORKAROUND, role.content_object does not work in migrations

I read all the posts/comments by @AlanCoding on the subject, but still, I cannot make it work. I browsed the Operator AWX CRD to see if there was an option to bypass that problem, with no success. I also searched throughout all the documentation I could find, but nothing on that topic.

As a solution, the only scenario that emerges right now for us is to dump the data, reinstall AWX on a empty database and then import the dump. But it sounds a bit risky and does not seem to be the legitimate choice.

I doubt it, but could it be related to the version of PostgreSQL ?

If I am missing something obvious, please enlighten me.

Thanks in advance.

2

Hi,

do you evaluate the migration to postgres 14/15?

In my lab I need to upgrade postgres then upgrade instance.

1 Like
3

I would also recommend upgrading your external PostgreSQL database as part of your AWX upgrade. As of AWX v24.0.0, the default postgres image uses PG15, and was upgraded from PG13. PG12 hasn’t been the default since… AWX v21.3.0.

Considering you are already at AWX v24.2.1, I suggest shutting down AWX, and performing a PG12 → PG13 → PG15 upgrade (I don’t think PostgreSQL supports directly upgrading from 12 to 15). Once the PG upgrade is complete, start AWX again, verify it’s working with the new database before continuing with the AWX upgrade.

1 Like
4

Ok, thanks for the replies @Denney-tech & @tanganellilore !

We were indeed planning a migration to PG 15, but we were not aware that it could block the path for the AWX migration.
We will do so and I will keep you informed.

Thanks again.

1 Like