AWX credential "Hushicorp vault signed SSH"

Get Help
1

I am using AWX tower to patch a linux VM and using Hushicorp vault signed SSH for connecting to all our linux vm. gathering facts is working and in my task i have performing pre-patch reboot after my vm got restart vm got disconnected error is “timeout waiting for last boot time”

can any one guide me on this if I need any config from my AWX tower side.

i am using AWX 23.2.0 and ansible core 2.14.11

my playbook looks:
- name: Pre-patching reboot
ansible.builtin.reboot:
reboot_timeout: “{{ patch_reboot_timeout | default(1500) }}”
tags:
- pre_patch_reboot

i am getting below Error in job output:
fatal: [10.144.35.54]: FAILED! => {

5520

“changed”: false,

5521

“elapsed”: 1505,

5522

“msg”: “Timed out waiting for last boot time check (timeout=1500)”,

5523

“rebooted”: true

5524

}

and i investicated target host log and i cant find anything:
Sep 9 08:53:08 athiru-as-1d sudo[2701]: ansible : PWD=/home/ansible ; USER=root ; COMMAND=/bin/sh -c ‘echo BECOME-SUCCESS-azjidbrafrhklffqsrqjgyshcnanbffd ; cat /proc/sys/kernel/random/boot_id’
Sep 9 08:53:08 athiru-as-1d sudo[2701]: pam_unix(sudo:session): session opened for user root by (uid=1001)
Sep 9 08:53:08 athiru-as-1d sudo[2701]: pam_unix(sudo:session): session closed for user root

2

anyone support on this pls

3

Unless Hashicorp* is rotating the authorized keys every reboot, then there’s no issue with that or AWX.

The reboot module is looking at the “last boot time” timestamp on the remote host, polling it periodically until the timestamp changes. There are a handful of scenarios where Ansible will timeout waiting for this value to change.

  1. The device is something like a Raspberry Pi 3B where this timestamp exists, but never changes. I forget the technical reasons for why, but I mention RPi3B specifically because I have experienced this.

    1A. To work around this issue, you have to send a shell reboot command to the host, then have ansible pause and wait for connection.

  2. The remote host is hung for some reason and cannot proceed with the reboot. This warrants a root-cause-analysis to determine why the host stalled on a reboot, and attempt to fix the root of the issue.

    2A. This might be addressed simply with reboot -f, but may lead to data loss, depending on the root issue.

  3. The host really needs more than 1500 seconds to reboot. Depending on a number of factors like hardware specs, system load, pending system patches, and what all needs to happen for a safe shutdown to complete; it could very well take a long time.

    3A. Generally your only option may be to extend the timeout. However, if the issue is how long it takes for a particular service to stop and start, then it might be easier to stop and start the service manually before and after a reboot.

1 Like
4

@thirumalairaja sorry to jump in, but I’m trying to accomplish (I believe) exactly what you already have, using the Hashicorp Vault to sign a public key to allow sshing into the target machines with ease.

Can you please share with me how did you proceed? I’m always getting ‘Invalid credentials’, as if the AWX is not considering my signed certificate.

I know that my configuration in Vault is working, because I did the process manually and was able to sshing into the target machine with no problems.

I following this old doc and perhaps that’s the problem:

If you could help me, I would really appreciate!