awx-operator backup : how can I remove no_log: true from postgres.yml

Archives AWX Project
1

Hi,
I am using K3s.

When I start a backup with awx-operator with :
kubectl apply -f backup/awxbackup.yaml
It keeps failing at the Postgres part of the ansible playbook postgres.yml
The logging of the containers it says:
“the output has been hidden due to the fact that ‘no_log: true’ was specified for this result”
When I connect to the manager container and view the ansible playbook postgres.yml

  • name: Get PostgreSQL configuration
    k8s_info:
    kind: Secret
    namespace: ‘{{ ansible_operator_meta.namespace }}’
    name: “{{ this_awx[‘resources’][0][‘status’][‘postgresConfigurationSecret’] }}”
    register: pg_config
    no_log: true

How can I change it?
crictl exec -it 225eb7d56b280 /bin/bash
I am user ansible.
Whereas the owner of the playbook is root.

Any help will be appreciated.

Regards Hans

2

Hi Hans,

I am interested in this answer as well because, from what I know, there is no way to remove the no_log part other than changing the operator playbook source and rebuilding the AWX operator container image.

Vincent

3

Hello Vincent,

I found this thread: https://groups.google.com/g/awx-project/c/9m7vESRa-Lw?pli=1
You can become root then in the container and end up in /opt/ansible
But unfortunately that does not help

You can write a file in /opt/ansible but if you descend into /opt/ansible/roles the tree is somehow read only.

k3s ctr task exec -t --exec-id abcd --user 0 bebe21f0114eee0547359512b1c7d527f2b7ec4ae0a41998d77d3b013964b0c4 bash

bash-4.4# ls
playbooks requirements.yml roles watches.yaml
bash-4.4# > x
bash-4.4# rm x
bash-4.4# cd roles/
bash-4.4# ls
backup installer restore
bash-4.4# > x
bash: x: No such file or directory
bash-4.4# ls -al
total 0
drwxr-xr-x 5 root root 52 May 2 18:45 .
drwxrwxr-x 1 ansible root 22 Jul 19 13:04 …
drwxr-xr-x 7 root root 93 May 2 18:44 backup
drwxr-xr-x 7 root root 93 May 2 18:44 installer
drwxr-xr-x 7 root root 93 May 2 18:44 restore
bash-4.4# pwd
/opt/ansible/roles

The only solution I saw is changing it from outside the container
I needed to change postgres.yml
Which is visable on the host as:

/var/lib/rancher/k3s/agent/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/58/fs/opt/ansible/roles/backup/tasks/postgres.yml

I ended up editing this file.
:open_mouth:

Regards Hans

4

Hi,

As of Operator 0.23.0 or later, “no_log” can be passed as a parameter for AWX, AWXBackup, and AWXRestore custom resources.
Therefore if upgrading Operator is acceptable for you, using “no_log” parameter for your custom resources is the most reasonable (and officially accepted) way, in my understanding.

If you want to debugging on older version on K3s anyway, as Vincent said, re-building Operator image is the one of the prefered way, but I like your approach :smiley:

FYI, since you seem to already have a Container ID, exploring /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/${CONTAINER_ID}/rootfs is more direct way than exploring snapshots under /var/lib, since the directory under /run/…/rootfs is actual mount point for snapshots under /var/lib for containerd.
For example, to force “no_log” disabled you could do the following.

sudo find /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/${CONTAINER_ID}/rootfs/opt/ansible/roles/backup -name “*.yml” | sudo xargs sed -i ‘s/no_log: true/no_log: false/g’

Regards,

5

Hi Vincent,

I discovered that adding the a verbosity to etc/ansible/ansible.cfg (located inside the operator container image) gives much more information.

Like this:

[default]
enable_task_debugger = True
verbosity = 4

Regards Hans

6

Hi Hans,

Thanks for sharing your solution, this will be useful for me to have !

Vincent