Parsing json files with ansible

Archives Ansible Project
1

I’m trying to parse the json output of a VMware DVSwitch which i’ve extracted using Ansible module community.vmware.vmware_dvs_portgroup_info

using this role, however i can’t seem to get to the portgroups, can only display the full json. What i’m i missing?

- hosts: 127.0.0.1
connection: local
become: yes
vars_files:
- answerfile_test.yml
- …/cred.yaml
vars:
dvs: “{{ lookup(‘file’, ‘result.json’) }}” tasks: - name: Simple debug.
debug:
msg: “{{ dvs.dvs_portgroup_info["DVS-Dev-Ork2"] }}”
tags:
- G2

Json output here:

{“changed”: false, “dvs_portgroup_info”: {“DVS-Dev-Ork2”: [{“portgroup_name”: “Test - 192.168.20.0_24 - VMotion - 1”, “num_ports”: 8, “dvswitch_name”: “DVS-Dev-Ork2”, “description”: null, “type”: “earlyBinding”, “teaming_policy”: {“policy”: “loadbalance_ip”, “inbound_policy”: true, “notify_switches”: true, “rolling_order”: false}, “port_policy”: {“block_override”: true, “ipfix_override”: false, “live_port_move”: false, “network_rp_override”: false, “port_config_reset_at_disconnect”: true, “security_override”: false, “shaping_override”: false, “traffic_filter_override”: false, “uplink_teaming_override”: false, “vendor_config_override”: false, “vlan_override”: false}, “network_policy”: {“forged_transmits”: false, “promiscuous”: false, “mac_changes”: false}, “vlan_info”: {“trunk”: false, “pvlan”: false, “vlan_id”: “20”}, “key”: “dvportgroup-14”}, {“portgroup_name”: “Test - 192.168.60.0_24 - Management”, “num_ports”: 15, “dvswitch_name”: “DVS-Dev-Ork2”, “description”: null, “type”: “earlyBinding”, “teaming_policy”: {“policy”: “loadbalance_ip”, “inbound_policy”: true, “notify_switches”: true, “rolling_order”: false}, “port_policy”: {“block_override”: true, “ipfix_override”: false, “live_port_move”: false, “network_rp_override”: false, “port_config_reset_at_disconnect”: true, “security_override”: false, “shaping_override”: false, “traffic_filter_override”: false, “uplink_teaming_override”: false, “vendor_config_override”: false, “vlan_override”: false}, “network_policy”: {“forged_transmits”: false, “promiscuous”: false, “mac_changes”: false}, “vlan_info”: {“trunk”: false, “pvlan”: false, “vlan_id”: “0”}, “key”: “dvportgroup-13”}, {“portgroup_name”: “Test - 192.168.30.0_24 - VSAN”, “num_ports”: 8, “dvswitch_name”: “DVS-Dev-Ork2”, “description”: null, “type”: “earlyBinding”, “teaming_policy”: {“policy”: “loadbalance_ip”, “inbound_policy”: true, “notify_switches”: true, “rolling_order”: false}, “port_policy”: {“block_override”: true, “ipfix_override”: false, “live_port_move”: false, “network_rp_override”: false, “port_config_reset_at_disconnect”: true, “security_override”: false, “shaping_override”: false, “traffic_filter_override”: false, “uplink_teaming_override”: false, “vendor_config_override”: false, “vlan_override”: false}, “network_policy”: {“forged_transmits”: false, “promiscuous”: false, “mac_changes”: false}, “vlan_info”: {“trunk”: false, “pvlan”: false, “vlan_id”: “25”}, “key”: “dvportgroup-15”}, {“portgroup_name”: “DVS-Dev-Ork2-DVUplinks-11”, “num_ports”: 18, “dvswitch_name”: “DVS-Dev-Ork2”, “description”: null, “type”: “earlyBinding”, “teaming_policy”: {“policy”: “loadbalance_srcid”, “inbound_policy”: true, “notify_switches”: true, “rolling_order”: false}, “port_policy”: {“block_override”: true, “ipfix_override”: false, “live_port_move”: false, “network_rp_override”: false, “port_config_reset_at_disconnect”: true, “security_override”: false, “shaping_override”: false, “traffic_filter_override”: false, “uplink_teaming_override”: false, “vendor_config_override”: false, “vlan_override”: false}, “network_policy”: {“forged_transmits”: true, “promiscuous”: false, “mac_changes”: false}, “vlan_info”: {“trunk”: true, “pvlan”: false, “vlan_id”: [“0-4094”]}, “key”: “dvportgroup-12”}]}, “failed”: false}

2

I’m trying to parse the json output of a VMware DVSwitch which i’ve extracted using Ansible module community.vmware.vmware_dvs_portgroup_info

using this role, however i can’t seem to get to the portgroups, can only display the full json. What i’m i missing?

- hosts: 127.0.0.1
connection: local
become: yes
vars_files:
- answerfile_test.yml
- …/cred.yaml
vars:
dvs: “{{ lookup(‘file’, ‘result.json’) }}” tasks: - name: Simple debug.
debug:
msg: “{{ dvs.dvs_portgroup_info["DVS-Dev-Ork2"] }}”
tags:
- G2

Json output here:

Is this the json output of the debug task or the content of the result.json file?

Also, why are you using an intermediate file?
Why not have that vmware_dvs_portgroup_info task be prior to the ones in the playbook you posted?

3

What result do you expect?

The attribute *DVS-Dev-Ork2* is a list of dictionaries. There are
many options on how to proceed. For example, you can iterate the list
and display *portgroups* along with the switch name

    - debug:
        msg: "{{ item.dvswitch_name }} {{ item.key }}"
      loop: "{{ dvs.dvs_portgroup_info['DVS-Dev-Ork2'] }}

Or, you can create a list of *portgroups*, e.g.

    - set_fact:
        portgroups: "{{ dvs.dvs_portgroup_info['DVS-Dev-Ork2']|
                        map(attribute='key')|list }}"

4

Hi Dick, it’s the content of the result.json

I do have a task that connects to the server and dumps the results.json.
For ease of troubleshooting it’s quicker to run it from a file but the end result is to get it directly from the mentioned task.

5

Hi Vladimir,

Thanks for your help! Basically I’m looking to gather portgroup_name and vlanid to use on the next task. What will be the best way to proceed?

6

Hi,

I would suggest the following solution to extract the required information:

  • hosts: 127.0.0.1
    connection: local
    vars:
    dvs: “{{ lookup(‘file’, ‘result.json’) }}”
    tasks:
  • name: Simple debug.
    debug:
    msg: “{{ dvs.dvs_portgroup_info["DVS-Dev-Ork2"] }}”
    tags:
  • G2

VERSION 1

  • name: Extract portgroup info
    set_fact:
    port_groups: “{{ dvs | json_query(query) }}”
    vars:
    query: |
    dvs_portgroup_info.*.{
    portgroup_name: portgroup_name,
    vlan_id: vlan_info.vlan_id
    }

  • debug:
    var: port_groups

VERSION 2

  • name: Extract portgroups alternative
    set_fact:
    port_groups_alt: “{{ (port_groups_alt | default()) + [port_group] }}”
    vars:
    port_group:
    portgroup_name: “{{ item.portgroup_name }}”
    vlan_id: “{{ item.vlan_info.vlan_id }}”
    loop: “{{ dvs.dvs_portgroup_info[‘DVS-Dev-Ork2’] }}”
    loop_control:
    label: “{{ item.portgroup_name }}”

  • debug:
    var: port_groups_alt

This playbook contains two alternative attempts. The first is using JMESPath via json_query and is a more declarative approach, if you will. The second is a plain looped set_fact.

Both have the same result: a list of dictionaries with portgroup_name and vlan_id:

TASK [Extract portgroup info] **************************************************************************************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1]

TASK [debug] *******************************************************************************************************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
“port_groups”: [
{
“portgroup_name”: “Test - 192.168.20.0_24 - VMotion - 1”,
“vlan_id”: “20”
},
{
“portgroup_name”: “Test - 192.168.60.0_24 - Management”,
“vlan_id”: “0”
},
{
“portgroup_name”: “Test - 192.168.30.0_24 - VSAN”,
“vlan_id”: “25”
},
{
“portgroup_name”: “DVS-Dev-Ork2-DVUplinks-11”,
“vlan_id”: [
“0-4094”
]
}
]
}

TASK [Extract portgroups alternative] ******************************************************************************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => (item=Test - 192.168.20.0_24 - VMotion - 1)
ok: [127.0.0.1] => (item=Test - 192.168.60.0_24 - Management)
ok: [127.0.0.1] => (item=Test - 192.168.30.0_24 - VSAN)
ok: [127.0.0.1] => (item=DVS-Dev-Ork2-DVUplinks-11)

TASK [debug] *******************************************************************************************************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
“port_groups_alt”: [
{
“portgroup_name”: “Test - 192.168.20.0_24 - VMotion - 1”,
“vlan_id”: “20”
},
{
“portgroup_name”: “Test - 192.168.60.0_24 - Management”,
“vlan_id”: “0”
},
{
“portgroup_name”: “Test - 192.168.30.0_24 - VSAN”,
“vlan_id”: “25”
},
{
“portgroup_name”: “DVS-Dev-Ork2-DVUplinks-11”,
“vlan_id”: [
“0-4094”
]
}
]
}

The json_query version would even work, if there are more than one object in dvs.dvs_portgroup_info.
The last entry may need additional handling as it contains a list of vlan_ids.

Cheers,
André

7

Hi André,

Thank you, this works perfectly except for one thing (which i’m starting to think its a bug in the ansible module)

I will consume the variables you helped me sort in another task as per below:
Now, if i use item.portgroup_name it fails, feels like its parsing the whole dict and not just the name.
However if i use any other attribute, say vlanid it will work. Any ideas?

This fails:

  • name: Create Segments
    nsxt_policy_segment:
    hostname: “{{ hostname }}”
    username: “{{ username }}”
    password: “{{ password }}”
    validate_certs: False
    display_name: “SEG-{{ item.portgroup_name }}”
    vlan_ids:
  • “{{ item.vlan_id }}”
    transport_zone_display_name: “{{ siteid }}-vlan-transport-zone”
    state: “{{ state }}”
    delegate_to: localhost
    loop: “{{ port_groups }}”
    tags:
  • CS

This works:

  • name: Create Segments
    nsxt_policy_segment:
    hostname: “{{ hostname }}”
    username: “{{ username }}”
    password: “{{ password }}”
    validate_certs: False
    display_name: “SEG-{{ item.vlan_id }}” <<- Only change
    vlan_ids:
  • “{{ item.vlan_id }}”
    transport_zone_display_name: “{{ siteid }}-vlan-transport-zone”
    state: “{{ state }}”
    delegate_to: localhost
    loop: “{{ port_groups }}”
    tags:
  • CS

Error:

TASK [Create Segments] ****************************************************************************************************************************************************************************************************
task path: /root/scripts/nsx-t/script4/nsxt-automation/1.yaml:38
ESTABLISH LOCAL CONNECTION FOR USER: root
EXEC /bin/sh -c ‘echo ~root && sleep 0’
EXEC /bin/sh -c ‘( umask 77 && mkdir -p “echo /root/.ansible/tmp”&& mkdir “echo /root/.ansible/tmp/ansible-tmp-1621604976.9795952-26415-266025818716905” && echo ansible-tmp-1621604976.9795952-26415-266025818716905=“echo /root/.ansible/tmp/ansible-tmp-1621604976.9795952-26415-266025818716905” ) && sleep 0’
Using module file /root/scripts/nsx-t/script4/nsxt-automation/library/nsxt_policy_segment.py
PUT /root/.ansible/tmp/ansible-local-261381voivn84/tmpcmt7bpon TO /root/.ansible/tmp/ansible-tmp-1621604976.9795952-26415-266025818716905/AnsiballZ_nsxt_policy_segment.py
EXEC /bin/sh -c ‘chmod u+x /root/.ansible/tmp/ansible-tmp-1621604976.9795952-26415-266025818716905/ /root/.ansible/tmp/ansible-tmp-1621604976.9795952-26415-266025818716905/AnsiballZ_nsxt_policy_segment.py && sleep 0’
EXEC /bin/sh -c ‘/usr/bin/python3 /root/.ansible/tmp/ansible-tmp-1621604976.9795952-26415-266025818716905/AnsiballZ_nsxt_policy_segment.py && sleep 0’
EXEC /bin/sh -c ‘rm -f -r /root/.ansible/tmp/ansible-tmp-1621604976.9795952-26415-266025818716905/ > /dev/null 2>&1 && sleep 0’
The full traceback is:
File “/tmp/ansible_nsxt_policy_segment_payload_5_a4rdcu/ansible_nsxt_policy_segment_payload.zip/ansible/module_utils/nsxt_base_resource.py”, line 713, in _send_request_to_API
ignore_errors=ignore_error, method=method, data=data)
File “/tmp/ansible_nsxt_policy_segment_payload_5_a4rdcu/ansible_nsxt_policy_segment_payload.zip/ansible/module_utils/policy_communicator.py”, line 143, in request
ca_path=self.ca_path)
File “/tmp/ansible_nsxt_policy_segment_payload_5_a4rdcu/ansible_nsxt_policy_segment_payload.zip/ansible/module_utils/urls.py”, line 1399, in open_url
unredirected_headers=unredirected_headers)
File “/tmp/ansible_nsxt_policy_segment_payload_5_a4rdcu/ansible_nsxt_policy_segment_payload.zip/ansible/module_utils/urls.py”, line 1304, in open
return urllib_request.urlopen(request, None, timeout)
File “/usr/lib64/python3.6/urllib/request.py”, line 223, in urlopen
return opener.open(url, data, timeout)
File “/usr/lib64/python3.6/urllib/request.py”, line 526, in open
response = self._open(req, data)
File “/usr/lib64/python3.6/urllib/request.py”, line 544, in _open
‘_open’, req)
File “/usr/lib64/python3.6/urllib/request.py”, line 504, in _call_chain
result = func(*args)
File “/tmp/ansible_nsxt_policy_segment_payload_5_a4rdcu/ansible_nsxt_policy_segment_payload.zip/ansible/module_utils/urls.py”, line 483, in https_open
return self.do_open(self._build_https_connection, req)
File “/usr/lib64/python3.6/urllib/request.py”, line 1349, in do_open
encode_chunked=req.has_header(‘Transfer-encoding’))
File “/usr/lib64/python3.6/http/client.py”, line 1254, in request
self._send_request(method, url, body, headers, encode_chunked)
File “/usr/lib64/python3.6/http/client.py”, line 1265, in _send_request
self.putrequest(method, url, **skips)
File “/usr/lib64/python3.6/http/client.py”, line 1127, in putrequest
raise InvalidURL(f"URL can’t contain control characters. {url!r} "
failed: [127.0.0.1] (item={‘portgroup_name’: ‘Test - 192.168.20.0_24 - VMotion - 1’, ‘vlan_id’: ‘20’}) => {
“ansible_loop_var”: “item”,
“changed”: false,
“invocation”: {
“module_args”: {
“achieve_subresource_state_if_del_parent”: false,
“address_bindings”: null,
“admin_state”: “UP”,
“advanced_config”: null,
“bridge_profiles”: null,
“ca_path”: null,
“connectivity_path”: null,
“create_or_update_subresource_first”: false,
“delete_subresource_first”: true,
“description”: null,
“dhcp_config_path”: null,
“do_wait_till_create”: false,
“domain_name”: null,
“enforcementpoint_id”: null,
“extra_configs”: null,
“hostname”: “nsx-dev-Ork2.local”,
“id”: null,
“l2_extension”: null,
“mac_pool_id”: null,
“metadata_proxy_paths”: null,
“nsx_cert_path”: null,
“nsx_key_path”: null,
“overlay_id”: null,
“password”: “VALUE_SPECIFIED_IN_NO_LOG_PARAMETER”,
“port”: 443,
“replication_mode”: “MTEP”,
“request_headers”: null,
“segment_ports”: null,
“site_id”: null,
“state”: “present”,
“subnets”: null,
“tags”: null,
“tier0_display_name”: null,
“tier0_id”: null,
“tier1_display_name”: null,
“tier1_id”: null,
“transport_zone_display_name”: “Ork2-vlan-transport-zone”,
“transport_zone_id”: null,
“username”: “admin”,
“validate_certs”: false,
“vlan_ids”: [
“20”
]
}
},
“item”: {
“portgroup_name”: “Test - 192.168.20.0_24 - VMotion - 1”,
“vlan_id”: “20”
},
“msg”: "Received URL can’t contain control characters. ‘/policy/api/v1/infra/segments/SEG-Test - 192.168.20.0_24 - VMotion - 1’ (found at least ’ ') from NSX Manager. Please try again. "
}

8

Got it working, turned out to be spaces in the portgroup_name values. ie. “Test - 192.168.20.0_24 - VMotion - 1”
I’ve renamed the portgroups and then it worked fine.

Any ideas how i can escape the dict values so i wont have to rename the portgroups?