AWX - Error { till now no solution} - please help

Archives AWX Project
1

Hi Friends,

i run firmware upgrade via AWX web container, but sometimes getting the below error: Till now I could not find a single solution. Request your big help here please …

TASK [esxi/upgrade : Wait for firmware job template] ***************************15:40:46126
{
“_ansible_parsed”: false,
“exception”: “Traceback (most recent call last):\n File "/root/.ansible/tmp/ansible-tmp-1599838846.25-48247545184850/AnsiballZ_tower_job_wait.py", line 113, in \n _ansiballz_main()\n File "/root/.ansible/tmp/ansible-tmp-1599838846.25-48247545184850/AnsiballZ_tower_job_wait.py", line 105, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File "/root/.ansible/tmp/ansible-tmp-1599838846.25-48247545184850/AnsiballZ_tower_job_wait.py", line 48, in invoke_module\n imp.load_module(‘main’, mod, module, MOD_DESC)\n File "/tmp/ansible_tower_job_wait_payload_8cSMv0/main.py", line 149, in \n File "/tmp/ansible_tower_job_wait_payload_8cSMv0/main.py", line 127, in main\n File "/var/lib/awx/venv/ansible/lib/python2.7/site-packages/tower_cli/models/base.py", line 880, in monitor\n result = client.get(job_endpoint).json()\n File "/var/lib/awx/venv/ansible/lib/python2.7/site-packages/requests/sessions.py", line 546, in get\n return self.request(‘GET’, url, **kwargs)\n File "/var/lib/awx/venv/ansible/lib/python2.7/site-packages/tower_cli/api.py", line 262, in request\n raise exc.ServerError('The Tower server sent back a server error. '\ntower_cli.exceptions.ServerError: The Tower server sent back a server error. Please try again later.\n”,
“_ansible_no_log”: false,
“_ansible_delegated_vars”: {
“ansible_delegated_host”: “localhost”,
“ansible_host”: “localhost”
},
“module_stderr”: “Traceback (most recent call last):\n File "/root/.ansible/tmp/ansible-tmp-1599838846.25-48247545184850/AnsiballZ_tower_job_wait.py", line 113, in \n _ansiballz_main()\n File "/root/.ansible/tmp/ansible-tmp-1599838846.25-48247545184850/AnsiballZ_tower_job_wait.py", line 105, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File "/root/.ansible/tmp/ansible-tmp-1599838846.25-48247545184850/AnsiballZ_tower_job_wait.py", line 48, in invoke_module\n imp.load_module(‘main’, mod, module, MOD_DESC)\n File "/tmp/ansible_tower_job_wait_payload_8cSMv0/main.py", line 149, in \n File "/tmp/ansible_tower_job_wait_payload_8cSMv0/main.py", line 127, in main\n File "/var/lib/awx/venv/ansible/lib/python2.7/site-packages/tower_cli/models/base.py", line 880, in monitor\n result = client.get(job_endpoint).json()\n File "/var/lib/awx/venv/ansible/lib/python2.7/site-packages/requests/sessions.py", line 546, in get\n return self.request(‘GET’, url, **kwargs)\n File "/var/lib/awx/venv/ansible/lib/python2.7/site-packages/tower_cli/api.py", line 262, in request\n raise exc.ServerError('The Tower server sent back a server error. '\ntower_cli.exceptions.ServerError: The Tower server sent back a server error. Please try again later.\n”,
“changed”: false,
“module_stdout”: “”,
“rc”: 1,
“msg”: “MODULE FAILURE\nSee stdout/stderr for the exact error”
}

Thanks for pointers…

sincerely
VJ

2

Look further up your logs. There isn’t much here to go on and the error is probably further up. Clean in the log file, run the job again with greater verbosity and parse each error it hits starting at the top and working your way down.

3

Thanks, Sossomons.

Today, I will run with verbosity as 4 (Connection Debug).
Requesting your suggestion, Is it good to maintain verbosity as 4 or 5 because 5 is WinRm Debug. All my automation is either network switches and vmware, vcenter based cluster tasks.

I checked AWX version it is showing as : AWX 3.0.0 but where ansible --version shows: ansible 2.7.11

sincerely
VJ

4

Vijay,

I’d recommend a setting of 5 for a single run against a host that failed and use that information to debug what all is happening. Once you have the failure contained in the log, export it and remove that run from the DB to save space.

If you keep your verbosity up, your logs will exponentially grow and blow out your DB size. I hadn’t paid attention to mine and wound up spending about a week getting things back under control. :frowning:

HTH,
Robert

5

Awesome, Thanks Robert.
Please share your inputs which is the target folder where, AWX task container stores all these verbose logs and where can i find if any troubleshooting doc available for AWX task container particularly

thanks,
sincerely
VJ