AWX human-readable Output Configuration (YAML)

Hello all!

I am attempting to enhance the human readability of template run outputs by configuring the stdout_callback plugin to YAML. In AWX, under “Settings/Jobs setting,” I add the following variable to “Extra Environment Variables”:

"ANSIBLE_STDOUT_CALLBACK": "yaml"

However, whenever I try to synchronize a project from my Git repository, the update fails with the following error message:

Identity added: /runner/artifacts/655/ssh_key_data (ssh awx keys)
ERROR! UnexpectedException, this is probably a bug: type 'NoneType' is not an acceptable base type
To see the full traceback, use -vvv

If I use "ANSIBLE_STDOUT_CALLBACK": "minimal" instead, the error disappears.

I suspect that the YAML callback plugin might not be present in the control-plane environment. With this idea in mind, I considered building an execution environment for the control-plane, similar to what I did for running my templates. However, it seems that this is not as straightforward and easily achievable as the former (it is not possible to select a different image in the AWX web for the “Control Plane Execution Environment”).

Is it possible to change the default stdout callback to YAML or something similar that can print job outputs in a more human-readable way?

Thanks for any attempt to help!

Peter

1 Like

All currently supported versions of ansible-core can be configured to use YAML formatted results with the default callback: ansible.builtin.default callback – default Ansible screen output — Ansible Documentation

2 Likes

Hello!

Thank you for your response!

Currently, I’m using ghcr.io/ansible/awx_devel, and I’ve experimented with various combinations and locations to set the callback variables. For instance, I also attempted using “ANSIBLE_CALLBACK_RESULT_FORMAT”: “yaml” under “Settings/Jobs/Environment Variables.” While this doesn’t trigger an error, it regrettably doesn’t have any impact on the output. :frowning:

Did you unset "ANSIBLE_STDOUT_CALLBACK": "yaml" when testing ANSIBLE_CALLBACK_RESULT_FORMAT?

It sounds like ANSIBLE_STDOUT_CALLBACK is configuring the callback, so the ANSIBLE_CALLBACK_RESULT_FORMAT environment var should also be capable of working. It should work with ansible-core 2.13 or later:

[shertel@fedora ansible]$ ANSIBLE_STDOUT_CALLBACK=minimal ansible localhost -m debug -v
No config file found; using defaults
localhost | SUCCESS => {
    "msg": "Hello world!"
}
[shertel@fedora ansible]$ ANSIBLE_STDOUT_CALLBACK=minimal ANSIBLE_CALLBACK_RESULT_FORMAT=yaml ansible localhost -m debug -v
No config file found; using defaults
localhost | SUCCESS => 
    msg: Hello world!
2 Likes

the yaml callback is part of the community.general collection, which I’d guess is not in the EE that is being used :person_shrugging:

1 Like

I’m not a full-time ansible engineer; I’m just doing my best to implement network configuration automation for JunOS and IOS-XR equipment. While using AWX, I had hoped that operational colleagues could benefit from the user interface. However, the output from ansible-playbook is at best like this:

In this instance, it’s a simple typo and an error, not a big deal to read it. However, often it is about displaying the result of rendered configurations, ranging from hundreds to several thousand lines, which is in a single line of text where non-printable control characters like LF/CR are represented as ‘\n’ neither readable nor valuable.

What I’m struggling to understand is why it seems so challenging for me to present the actual job output of AWX in a format that is human-readable. Am I misunderstanding a fundamental principle of AWX/Ansible? Isn’t the stdout of AWX/ansible-playbook mostly interpreted by humans?

If the ansible.builtin default callback is configured with the yaml format as suggested by @flowerysong, the output looks like this:

$ ANSIBLE_STDOUT_CALLBACK=default ANSIBLE_CALLBACK_RESULT_FORMAT=yaml ansible-playbook p.yml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] ********************************************************************************************************************************************************************************************************************

TASK [command] **********************************************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => 
    msg: |-
        The task includes an option with an undefined variable. The error was: 'something_undefined' is undefined. 'something_undefined' is undefined

        The error appears to be in '/home/shertel/ansible/p.yml': line 4, column 7, but may
        be elsewhere in the file depending on the exact syntax problem.

        The offending line appears to be:

          tasks:
            - command: "{{ something_undefined }}"
              ^ here
        We could be wrong, but this one looks like it might be an issue with
        missing quotes. Always quote template expression brackets when they
        start a value. For instance:

            with_items:
              - {{ foo }}

        Should be written as:

            with_items:
              - "{{ foo }}"

PLAY RECAP **************************************************************************************************************************************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

vs,

$ ANSIBLE_STDOUT_CALLBACK=default ANSIBLE_CALLBACK_RESULT_FORMAT=json ansible-playbook p.yml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] ********************************************************************************************************************************************************************************************************************

TASK [command] **********************************************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'something_undefined' is undefined. 'something_undefined' is undefined\n\nThe error appears to be in '/home/shertel/ansible/p.yml': line 4, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n  tasks:\n    - command: \"{{ something_undefined }}\"\n      ^ here\nWe could be wrong, but this one looks like it might be an issue with\nmissing quotes. Always quote template expression brackets when they\nstart a value. For instance:\n\n    with_items:\n      - {{ foo }}\n\nShould be written as:\n\n    with_items:\n      - \"{{ foo }}\"\n"}

PLAY RECAP **************************************************************************************************************************************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

I don’t see an open issue for AWX not working with callbacks, besides Provide a way to configure a custom stdout callback per job template · Issue #14281 · ansible/awx · GitHub because the stdout callback can only be configured globally with env vars. Unless you have something conflicting configured, I don’t see how ANSIBLE_CALLBACK_RESULT_FORMAT set the same way wouldn’t work with AWX when using the default or minimal ansible.builtin callbacks :confused:

1 Like

Any suggestion how to debug this?

I imagine this might be more involved than you’re interested in, but you could pull down the EE that is running your jobs and run it locally to try to reproduce the issue outside of AWX / verify the community.general collection is in the EE. If not, you could create your own EE with the community.general collection included and use that for your AWX jobs. Getting started with Execution Environments is a great resource if you want to go down that path.

1 Like

I have already performed this task using the default AWX EE configuration from https://github.com/ansible/awx-ee, incorporating the following additional collections, community.general included:

[c64@localhost ansible-builder]$ cat execution-environment.yml
---
version: 3
images:
  base_image:
    name: quay.io/centos/centos:stream9
dependencies:
  ansible_core:
    # Require minimum of 2.15 to get ansible-inventory --limit option
    package_pip: ansible-core>=2.15.0rc2,<2.16
  ansible_runner:
    package_pip: ansible-runner
  galaxy: |
    ---
    collections:
      - name: awx.awx
      - name: community.vmware
      - name: ansible.posix
      - name: ansible.windows
      - name: redhatinsights.insights
      - name: ansible.netcommon
      - name: ansible.utils
      - name: juniper.device
      - name: junipernetworks.junos
      - name: netbox.netbox
      - name: cisco.aci
      - name: cisco.ios
      - name: cisco.nxos
      - name: f5networks.f5_modules
      - name: grafana.grafana
      - name: infoblox.nios_modules
      - name: community.network
      - name: community.general
  system: |
    git-core [platform:rpm]
    python3.9-devel [platform:rpm compile]
    libcurl-devel [platform:rpm compile]
    krb5-devel [platform:rpm compile]
    krb5-workstation [platform:rpm]
    subversion [platform:rpm]
    subversion [platform:dpkg]
    git-lfs [platform:rpm]
    sshpass [platform:rpm]
    rsync [platform:rpm]
    epel-release [platform:rpm]
    python-unversioned-command [platform:rpm]
    unzip [platform:rpm]
  python: |
    git+https://github.com/ansible/ansible-sign
    ncclient
    paramiko
    pykerberos
    pyOpenSSL
    pypsrp[kerberos,credssp]
    pywinrm[kerberos,credssp]
    toml
    pexpect>=4.5
    python-daemon
    pyyaml
    six
    junos-eznc
    jxmlease
    jsnapy
    jmespath
additional_build_steps:
  append_base:
    - RUN $PYCMD -m pip install -U pip
  append_final:
    - COPY --from=quay.io/ansible/receptor:devel /usr/bin/receptor /usr/bin/receptor
    - RUN mkdir -p /var/run/receptor
    - RUN git lfs install --system

However, it has not resulted in any change in behavior.

Have you tried without AWX? You could try to debug using the ansible.builtin.config lookup:

# OPTION - (VALUE, CONFIGURED-VIA)

- debug: msg="stdout callback - {{ stdout_callback }}"
  vars:
    stdout_callback: "{{ lookup('config', 'DEFAULT_STDOUT_CALLBACK', show_origin=true) }}"

- debug: msg="default callback result format - {{ result_format }}"
  vars:
    result_format: "{{ lookup('config', 'result_format', plugin_type='callback', plugin_name='default', show_origin=true) }}"

This is what is should look like with no configured callback (‘default’ is used) and yaml set via env var:

$ ANSIBLE_CALLBACK_RESULT_FORMAT=yaml ansible-playbook playbook.yml
...
TASK [debug] ************************************************************************************************************************************************************************************************************************
ok: [localhost] => 
    msg: stdout callback - ('default', None)

TASK [debug] ************************************************************************************************************************************************************************************************************************
ok: [localhost] => 
    msg: 'default callback result format - (''yaml'', ''env: ANSIBLE_CALLBACK_RESULT_FORMAT'')'
1 Like

Thank you for these suggestion!

Indeed the config lookup “trick” gave some hint about the issue:

So for unknown reason the DEFAULT_STDOUT_CALLBACK variable is sticking to “awx_display” despite being set to “default”.

(venv) ❯ ~/.local/bin/ansible-navigator run debug_config_vars.yml --eei ghcr.io/ansible-community/community-ee-base:latest -m stdout --senv ANSIBLE_CALLBACK_RESULT_FORMAT=yaml DEFAULT_STDOUT_CALLBACK=default
-------------------------------------------------------------------------------------
Execution environment image and pull policy overview
-------------------------------------------------------------------------------------
Execution environment image name:     ghcr.io/ansible-community/community-ee-base:latest
Execution environment image tag:      latest
Execution environment pull arguments: None
Execution environment pull policy:    tag
Execution environment pull needed:    True
-------------------------------------------------------------------------------------
Updating the execution environment
-------------------------------------------------------------------------------------
Running the command: podman pull ghcr.io/ansible-community/community-ee-base:latest
Trying to pull ghcr.io/ansible-community/community-ee-base:latest...
Getting image source signatures
Copying blob 00fcfe03da1d skipped: already exists
Copying blob 718a00fe3212 skipped: already exists
Copying blob b5dab7825985 skipped: already exists
Copying blob a4888f00cc3e skipped: already exists
Copying blob f594bcb86bf2 skipped: already exists
Copying blob b78e7b7ffc9f skipped: already exists
Copying blob 0eeac9cdd3ed skipped: already exists
Copying blob 135fe6b66968 skipped: already exists
Copying blob 553abf1605d4 skipped: already exists
Copying blob a16eb863c593 skipped: already exists
Copying blob 90168653f9f0 skipped: already exists
Copying blob 861d994d3ea2 skipped: already exists
Copying blob 6a01cb961aa8 skipped: already exists
Copying config 2cf8285eaa done   |
Writing manifest to image destination
2cf8285eaa5682d20288eaa077ce52c2da5a0607586e0330402e614a9ccbd2c1
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'

PLAY [Debug hostvars] **********************************************************

TASK [debug] *******************************************************************
ok: [localhost] => {
    "msg": "stdout callback - ('awx_display', None)"
}

TASK [debug] *******************************************************************
ok: [localhost] => {
    "msg": "default callback result format - ('yaml', 'env: ANSIBLE_CALLBACK_RESULT_FORMAT')"
}

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Do all execution environments overwrite the DEFAULT_STDOUT_CALLBACK?

It appears that this is a somewhat obscure matter, with limited awareness or concern among users. Consequently, I infer that alternative methods must exist for rendering the output of AWX playbooks/templates in a human-readable and practical format.

I don’t use AWX, but I believe you can just click on the result to see it in a more useful format. The default display is just intended to give you an overview, not show every piece of task information that is available.

@c64
Hi, in my understanding of the current implementation of ansible-runner and AWX,

  • ANSIBLE_STDOUT_CALLBACK=community.general.yaml may work, but we have to install community.general to the EE image
  • ANSIBLE_CALLBACK_RESULT_FORMAT=yaml may not work, but we can patch awx-ee to support this

Here are steps for these two ways:

Use ANSIBLE_STDOUT_CALLBACK=yaml

  1. Build following Dockerfile and push to the container registry. (Of cource you can build new EE with ansible-builder, but building following simple Dockerfile is the quickest way to add collections to the existing EE image)
    FROM quay.io/ansible/awx-ee:latest
    
    USER root
    RUN ansible-galaxy collection install -p 
    /usr/share/ansible/collections/ansible_collections community.general
    USER 1000
    
  2. Specify control_plane_ee_image with your new EE image to your AWX CR (This will update Control Plane Execution Environment in your AWX):
    spec:
      ...
      control_plane_ee_image: registry.example.com/ansible/awx-ee-community-general:latest
    
  3. In AWX, update Execution Environment to point your new EE image, and add Extra Environment Variables with "ANSIBLE_STDOUT_CALLBACK": "community.general.yaml"

Use ANSIBLE_CALLBACK_RESULT_FORMAT=yaml

  1. Build following Dockerfile and push to the container registry
    FROM quay.io/ansible/awx-ee:latest
    
    USER root
    RUN sed -i 's/      - default_callback/      - default_callback\n      - result_format_callback/g' /usr/local/lib/python3.9/site-packages/ansible_runner/display_callback/callback/awx_display.py
    USER 1000
    
  2. Specify control_plane_ee_image with your new EE image to your AWX CR:
    spec:
      ...
      control_plane_ee_image: registry.example.com/ansible/awx-ee-yaml-patch:latest
    
  3. In AWX, update Execution Environment to point your new EE image, and add Extra Environment Variables with "ANSIBLE_CALLBACK_RESULT_FORMAT": "yaml"

Additional Info

  • Ansible Runner forces stdout callback to be overridden by awx_display.
  • However, awx_display is loaded dynamically with inheriting the originally specified stdout callback.
  • Thus, although the lookup('config') displays awx_display, internally the original stdout callback is still in operation (in fact, if you specify minimal, the job logs will be minimized, but the lookup('config') result should still be awx_display). This is why my first suggestion works.
  • However, awx_display has not been updated to enable the newly implemented result_format option. Patching awx_display to support result_format is my second suggestion. There is an open issue for this topic.

Since Extra Environment Variables is a global setting, you can instead place ansible.cfg with stdout_callback = community.general.yaml orcallback_result_format = yaml in the root of your project and get the same result. In this case, control_plane_ee_image does not necessarily need to be configured, if it’s okay that the project sync logs are in JSON.

(UPDATE: Sorry, ansible.cfg works only for callback_result_format = yaml. stdout_callback has to be defined via environment variable)

Hope this helps :smiley:

3 Likes

image
It would be nice if we can choose yaml format in output job. I hope awx support this feature in the future :blush: