Not able to execute a python script from AWX

Archives AWX Project
1

Hi All,

Hope you all are doing good.

I am trying to execute a python script as part of an ansible role. But AWX dont execute the script and there are no errors on job run as well.

Also if we try to run the playbook via command ansible-playbook then the script does get executed.

Can you please help me, as in why it dont works with AWX.

Below is the code for the same.

  • name: Execute the kafka cluster script
    shell:
    cmd: python /opt/icapi/ICAPI-DataDog/ic2datadog.py kafka_cluster_config.json &

Regards,
Ankur Porwal

2

Hi Ankur,

Hi All,

Hope you all are doing good.

I am trying to execute a python script as part of an ansible role. But AWX dont execute the script and there are no errors on job run as well.

Could you share your output when the playbook runs with verbosity enabled to 3 or 4? I find strange you get no error whatsoever…

I suspect something is wrong with your Execution Environment and the binary is not present for Python as it should. This should show in the playbook run logs.

HTH,

Vincent

3

Thanks Vincent for taking time to reply on the issue.

Please find below the logs from AWX after enabling verbosity.

Just to update you that am using AWX version 15.0.0 and ansible version as 2.9.13

TASK [srev_instaclustr_datadog_agent : Execute the kafka cluster script] *******
task path: /tmp/awx_1042479_szxroolb/project/roles/srev_instaclustr_datadog_agent/tasks/main.yml:64
Monday 06 December 2021 09:43:48 +0000 (0:00:01.634) 0:04:44.382 *******
<xx.3.21.224> ESTABLISH SSH CONNECTION FOR USER: ec2-user
<xx.3.21.224> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o Port=2222 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“ec2-user”’ -o ConnectTimeout=10 -o ControlPath=/tmp/awx_1042479_szxroolb/cp/f80bf129af xx.3.21.224 ‘/bin/sh -c ‘"’“‘echo ~ec2-user && sleep 0’”’“‘’
<xx.3.21.224> (0, b’/home/ec2-user\n’, b’')
<xx.3.21.224> ESTABLISH SSH CONNECTION FOR USER: ec2-user
<xx.3.21.224> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o Port=2222 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“ec2-user”’ -o ConnectTimeout=10 -o ControlPath=/tmp/awx_1042479_szxroolb/cp/f80bf129af xx.3.21.224 '/bin/sh -c '”‘"’( umask 77 && mkdir -p “echo /home/ec2-user/.ansible/tmp”&& mkdir “echo /home/ec2-user/.ansible/tmp/ansible-tmp-1638783828.3375993-1054-195071821294354” && echo ansible-tmp-1638783828.3375993-1054-195071821294354=“echo /home/ec2-user/.ansible/tmp/ansible-tmp-1638783828.3375993-1054-195071821294354” ) && sleep 0’“'”‘’
<xx.3.21.224> (0, b’ansible-tmp-1638783828.3375993-1054-195071821294354=/home/ec2-user/.ansible/tmp/ansible-tmp-1638783828.3375993-1054-195071821294354\n’, b’‘)
Using module file /usr/lib/python3.6/site-packages/ansible/modules/commands/command.py
<xx.3.21.224> PUT /var/lib/awx/.ansible/tmp/ansible-local-311004e45k68c/tmpq32tuwr6 TO /home/ec2-user/.ansible/tmp/ansible-tmp-1638783828.3375993-1054-195071821294354/AnsiballZ_command.py
<xx.3.21.224> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o Port=2222 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“ec2-user”’ -o ConnectTimeout=10 -o ControlPath=/tmp/awx_1042479_szxroolb/cp/f80bf129af ‘[xx.3.21.224]’
<xx.3.21.224> (0, b’sftp> put /var/lib/awx/.ansible/tmp/ansible-local-311004e45k68c/tmpq32tuwr6 /home/ec2-user/.ansible/tmp/ansible-tmp-1638783828.3375993-1054-195071821294354/AnsiballZ_command.py\n’, b’‘)
<xx.3.21.224> ESTABLISH SSH CONNECTION FOR USER: ec2-user
<xx.3.21.224> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o Port=2222 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“ec2-user”’ -o ConnectTimeout=10 -o ControlPath=/tmp/awx_1042479_szxroolb/cp/f80bf129af xx.3.21.224 ‘/bin/sh -c ‘"’“‘chmod u+x /home/ec2-user/.ansible/tmp/ansible-tmp-1638783828.3375993-1054-195071821294354/ /home/ec2-user/.ansible/tmp/ansible-tmp-1638783828.3375993-1054-195071821294354/AnsiballZ_command.py && sleep 0’”’"’’
<xx.3.21.224> (0, b’‘, b’‘)
<xx.3.21.224> ESTABLISH SSH CONNECTION FOR USER: ec2-user
<xx.3.21.224> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o Port=2222 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“ec2-user”’ -o ConnectTimeout=10 -o ControlPath=/tmp/awx_1042479_szxroolb/cp/f80bf129af -tt xx.3.21.224 ‘/bin/sh -c ‘"’“'sudo -H -S -n -u root /bin/sh -c '”’"’“'”‘"’“'”‘“‘echo BECOME-SUCCESS-oojnlxxkwnsxnophnkjearkdnewytmln ; /usr/bin/python /home/ec2-user/.ansible/tmp/ansible-tmp-1638783828.3375993-1054-195071821294354/AnsiballZ_command.py’”’“'”‘"’“'”‘"’“’ && sleep 0’”‘"’’
Escalation succeeded
<xx.3.21.224> (0, b’\r\n{“changed”: true, “end”: “2021-12-06 09:44:09.745473”, “stdout”: “”, “cmd”: “python /opt/icapi/ICAPI-DataDog/ic2datadog.py kafka_cluster_config.json &”, “rc”: 0, “start”: “2021-12-06 09:44:08.742850”, “stderr”: “”, “delta”: “0:00:01.002623”, “invocation”: {“module_args”: {“creates”: null, “executable”: null, “_uses_shell”: true, “strip_empty_ends”: true, “_raw_params”: “python /opt/icapi/ICAPI-DataDog/ic2datadog.py kafka_cluster_config.json &”, “removes”: null, “argv”: null, “warn”: true, “chdir”: null, “stdin_add_newline”: true, “stdin”: null}}}\r\n’, b’Shared connection to xx.3.21.224 closed.\r\n’)
<xx.3.21.224> ESTABLISH SSH CONNECTION FOR USER: ec2-user
<xx.3.21.224> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o Port=2222 -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“ec2-user”’ -o ConnectTimeout=10 -o ControlPath=/tmp/awx_1042479_szxroolb/cp/f80bf129af xx.3.21.224 ‘/bin/sh -c ‘"’“‘rm -f -r /home/ec2-user/.ansible/tmp/ansible-tmp-1638783828.3375993-1054-195071821294354/ > /dev/null 2>&1 && sleep 0’”’"‘’
<xx.3.21.224> (0, b’‘, b’')
changed: [xx.3.21.224] => {
“changed”: true,
“cmd”: “python /opt/icapi/ICAPI-DataDog/ic2datadog.py kafka_cluster_config.json &”,
“delta”: “0:00:01.002623”,
“end”: “2021-12-06 09:44:09.745473”,
“invocation”: {
“module_args”: {
“_raw_params”: “python /opt/icapi/ICAPI-DataDog/ic2datadog.py kafka_cluster_config.json &”,
“_uses_shell”: true,
“argv”: null,
“chdir”: null,
“creates”: null,
“executable”: null,
“removes”: null,
“stdin”: null,
“stdin_add_newline”: true,
“strip_empty_ends”: true,
“warn”: true
}
},
“rc”: 0,
“start”: “2021-12-06 09:44:08.742850”,
“stderr”: “”…

4

Hi Ankur,

Thanks for the logs. The bottom appears to be cut though, just after the “stderr” part there is an ellipsis. Is that on purpose?

Another question: what makes you think the script was not executed? I see the return code is zero which means the call went well.

One final note: do you have the same thing if you don’t include the ampersand character (i.e. make the call blocking)?

HTH,

Vincent

5

Hi Vincent,

For 1st question, that ellipsis is a copy and paste thing.

For 2nd question, I checked the process on remote machine and I didnt found any python process running.

For 3rd question, I need this script to be running 24x7, Hence am running it in background with ampersand.

Also, just to confirm, can it be because we have different python version on AWX and remote machines ?

6

Hi Vincent,

Just to cross check, is there any workaround for this use case i.e. “to run a python script in background from AWX” ?

Do you think using a cron job for the same makes sense ?

7

Hi Ankur,

Please see my answers below.

Hi Vincent,

For 1st question, that ellipsis is a copy and paste thing.

Ok, this was to make sure I had not missed anything in this output.

For 2nd question, I checked the process on remote machine and I didnt found any python process running.

You mean you logged on the remote node which the playbook targeted and checked there?

For 3rd question, I need this script to be running 24x7, Hence am running it in background with ampersand.

If that is the case, I think you should write a systemd service unit file instead. This would provide system-level management w/ automated restarts, etc

Also, just to confirm, can it be because we have different python version on AWX and remote machines ?

It could be but there are many more things to take into account: the environment when run on AWX can be very different than from a normal machine (Cf the concept of execution environments and what you included in its associated image).

As for a workaround (cf your other post), the answer is a systemd service unit file to install and activate. A cron job could also be used although a systemd timer is a better/more modern solution

HTH,

Vincent