I had a working installation of AWX 22.3.0, but figured it’s over a year old; I should update to something newer.
I tried following various docs:
Backup role allowed me to create a backup, though I’m not super confident that I’d be able to restore it.
Upgrading doc advised deleting old Operator, so I did that.
It then suggested using Basic Install to install the newer operator (I’m using 2.19.1).
I ran make deploy, created kustomization.yml with my AWX deployment name and tag, and include the awx-demo.yml
Ran kubectl apply -k .
It seemed to get rid of my old awx-demo pods (task, web, etc.) and start its install process
Install process creates a Postgres 15 pod (old DB was 13). New Postgres pod is stuck in “Pending” with “mkdir: cannot create directory ‘/var/lib/pgsql/data/userdata’: Permission denied”
awx-manager logs seem to be looping, trying to install, “"StatefulSet" "awx-demo-postgres-15": Timed out waiting on resource”
@kurokobo Thanks! I did this, plus a few more steps and was able to get it working, with a few snags along the way. Full detail:
Added postgres_data_volume_init: true to the awx-demo.yml and then re-ran kubectl apply -k .
Evicted the old operator pod and the old postgres-15 pod. (Note that my old install was using a postgres-13 pod; I evicted the “new” postgres-15 pod that was failing.)
It ran for a while, deleted the old postgres-13 pod, and then threw a different error, which was hidden due to no_log: true
Added no_log: false to awx-demo.yml and then re-ran kubectl apply -k and evicted the operator pod.
New error log showed “pg_dump: error: could not translate host name "awx-demo-postgres-13"” (of course this was because the install had deleted it earlier)
Removed the old_postgres_configuration_secret from awx-demo.yml and deleted the Secret “awx-demo-old-postgres-configuration” and then re-ran kubectl apply -k and evicted the operator pod.
From there, it seemed to succeed.
Thinking about how the docs might be updated to avoid someone else getting stuck with this. Thoughts on adding, in Basic Install doc mention of postgres_data_volume_init: true ?
If that seems like a good practice for all installs, I’d be happy to make a git PR to add some detail on that to the docs. Thoughts?
The parameter postgres_data_volume_init is not necessarily required for all users. For example, it does not required on OpenShift, and even on Kubernetes, whether the issue arises depends on how the storage providing the PV behaves (how the default permissions of the PV are set). This is also mentioned in the documentation that explains this parameter. That is why this parameter is false by default.
I think adding more description about this setting in the documentation is a good idea, but it might be more natural to add it to the troubleshooting section rather than the basic install section.
