json_query/jmespath filtering question

Trying to wrap my head about how to query json results from a URI call & could use some guidance. I am guessing there are plenty in this group which this will be an easy question for.

I am registering the json response from a URI call to our backup system to ‘snapshotlist’

Here is an example chunk of the json:

“json”: {
“vmwareTagList”: ,
“isCorrupt”: false,
“snapshotDiskDetails”: [
{
“virtualDiskId”: “23478e90-0228-4tab-ba99-d3894f94c8c837-vm-98-2001”,
“deviceKey”: 2001,
“size”: 214748364800,
“id”: “83fecf84-26db-42db-9f33-ec7e10b5f736”,
“fileName”: “[DC1_LINUX_VM_STORAGE] host2restore/host2restore_1.vmdk”
},
{
“virtualDiskId”: “23478e90-0228-4tab-ba99-d3894f94c8c837-vm-98-2002”,
“deviceKey”: 2002,
“size”: 214748364800,
“id”: “1a9c094f-f19b-4ea7-1406-ab145b0ec2f1”,
“fileName”: “[DC1_LINUX_VM_STORAGE] host2restore/host2restore_2.vmdk”
},
{
“virtualDiskId”: “23478e90-0228-4tab-ba99-d3894f94c8c837-vm-98-2000”,
“deviceKey”: 2000,
“size”: 42949672960,
“id”: “83182b0a-bc21-4ef8-8582-7ec95465380b”,
“fileName”: “[DC1_LINUX_VM_STORAGE] host2restore/host2restore.vmdk”
}
],
“date”: “2020-03-10T05:42:32.675Z”,
“cloudState”: 0,

I am trying to get the value of “id” for the object with fileName that ends with host2retore_1.vmdk

This is what I currently have to try to pull that value out based on some example I found on a blog somewhere. However I get errors. Anyone able to help me figure out what I am doing wrong?

  • name: set variable for disk_snap_id
    set_fact:
    disk_snap_id: “{{ json_query(‘snapshotdetails.json.snapshotDiskDetails[?fileName.contains(@, ‘host2restore_1.vmdk’)].virtualDiskId’) }}”

You need to filter a variable through json_query.

Also, I don’t see any key called ‘snapshotdetails’.
And that json string should be filtered through from_json, ideally.

It looks like your “json” var isn’t really what you are using.
If I shuffle things around slightly, then below playbook works:

Oops I said the wrong register variable name in my question. snapshotlist is the registered output of a previous task, snapshotdetails is the register of the task right before this particular task. the chunk of json I pasted in is pulled from the previous task’s output.
I was previously able to use snapshotdetails.json.snapshotDiskDetails[1].virtualDiskId to get a value in the play, it was the filtering that I have been struggling with.

Thanks for the reply I will give that a shot.

Still struggling to get this working. Posting my entire playbook in case it can help understand what I might be doing wrong:

  • hosts: ‘{{inventory_hostname}}’
    gather_facts: no
    vars:
    cluser_ip: “{{ rubrik_hostname }}”

tasks:

  • name: login to rubrik to generate token
    uri:
    url: https://{{ cluser_ip }}/api/v1/session # URL to generate an auth token to be used by subsequent calls
    force_basic_auth: yes
    user: ‘{{ lookup(“env”, “rubrik_username”) }}’
    password: ‘{{ lookup(“env”, “rubrik_password”) }}’
    method: POST
    status_code: 200
    return_content: yes
    validate_certs: yes
    register: authtoken # token is available as authtoken.json.token
    delegate_to: localhost
    become: no

  • name: get target vm id
    uri:
    url: https://{{ cluser_ip }}/api/v1/vmware/vm?name={{ inventory_hostname_nfq }}
    headers:
    Content-Type: “application/json”
    Authorization: “Bearer {{ authtoken.json.token }}” # pass authentication token from earlier rather than basic auth
    method: GET
    status_code: 200
    return_content: yes
    validate_certs: yes
    register: vminfo
    delegate_to: localhost
    become: no

  • name: set variable for target_vm_id
    set_fact:
    target_vm_id: “{{ vminfo.json.data[0].id }}”

  • name: get list of snapshots
    uri:
    url: https://{{ cluser_ip }}/api/v1/vmware/vm/{{ source_vm_id }}/snapshot
    headers:
    Content-Type: “application/json”
    Authorization: “Bearer {{ authtoken.json.token }}” # pass authentication token from earlier rather than basic auth
    method: GET
    status_code: 200
    return_content: yes
    register: snapshotlist
    delegate_to: localhost
    become: no

  • name: get info on the snapshot to be used
    uri:
    url: https://{{ cluser_ip }}/api/v1/vmware/vm/snapshot/{{ snapshotlist.json.data[0].id }}
    headers:
    Content-Type: “application/json”
    Authorization: “Bearer {{ authtoken.json.token }}” # pass authentication token from earlier rather than basic auth
    method: GET
    status_code: 200
    return_content: yes
    register: snapshotdetails
    delegate_to: localhost
    become: no

  • name: set variable for disk_snap_id
    set_fact:
    disk_snap_id: “{{ snapshotdetails | from_json | json_query(‘json.snapshotDiskDetails[?fileName.contains(@, ‘host2restore_1’)].virtualDiskId’) }}”

Error I am getting currently:

TASK [set variable for disk_snap_id] *******************************************
task path: /tmp/awx_56722_7ypjkkvu/project/oracle_refresh_fixorder.yml:67
fatal: [host2]: FAILED! => {
“msg”: “template error while templating string: expected token ‘,’, got ‘host2restore_1’. String: {{ snapshotdetails | from_json | json_query(‘json.snapshotDiskDetails[?fileName.contains(@, ‘host2restore_1’)]’) }}”
}

I also attempted setting the actual query string in a separate var, and passing that to the json_query as I have seen in many examples but that also did not work. not sure what I am doing wrong. A few permutations I have tried have run successfully but then returned a blank value. As you can see from the playbook snapshotdetails is a registered variable from the previous task.

The query for json_query is between single quotes. But you also use
single quotes to quote 'host2restore_1' - that doesn't work.
You need to use backticks instead.
If you look closely you can see that those were in the previous
example. I agree that it's not easy to spot :slight_smile:

Dick

Yup I absolutely did not notice that, thank you for pointing it out. now I am getting the following error:

fatal: [host2]: FAILED! => {
“msg”: “Unexpected templating type error occurred on ({{ snapshotdetails | from_json | json_query(‘json.snapshotDiskDetails[?fileName.contains(@, host2restore_1)].virtualDiskId’) }}): expected string or buffer”
}

Can you put a debug task between the uri and set_fact tasks and post
what that outputs?

- debug: var=snapshotdetails

If you put this in your ansible.cfg then it's more readable:

stdout_callback = yaml
bin_ansible_callbacks = True

managed to get it working with this:

  • name: set variable for disk_snap_id
    set_fact:
    disk_snap_id: “{{ snapshotdetails | to_json | from_json | json_query(‘json.snapshotDiskDetails[?fileName.contains(@, host2restore_1)].virtualDiskId’) }}”

after finding this:

https://stackoverflow.com/questions/48318708/unexpected-templating-type-error-occured-on-ec2-instances-from-json-exp