How to create with_items, on a submember?

The output (i.e. module: ec2_snapshot) of a registered variable (i.e. ec2_snap) returns multiple members.

Playbook is:

— # Take Snapshot of EC2s

  • name: Take snapshot of EC2s
    hosts: local
    become: no
    connection: local
    gather_facts: yes
    vars:
    region: “us-east-1”
    awsec2id: [“i-037b596065189XXXXX”,“i-04452f010e4cXXXXX”]
    devname: “/dev/sda1”
    date: “{{ lookup(‘pipe’, ‘date +%Y/%m/%d-%H:%M’) }}”
    tasks:

  • name: Create Root snapshots
    ec2_snapshot:
    region: ‘{{ region }}’
    instance_id: ‘{{ item }}’
    device_name: ‘{{ devname }}’
    description: ‘snapshot of /dev/sda1 from {{ item }} taken {{ date }}’
    state: present
    snapshot_tags:
    frequency: daily
    source: ‘{{ devname }}’
    register: ec2_snap
    with_items: ‘{{ awsec2id }}’

  • debug: var=ec2_snap

  • debug: var=ec2_snap.results[0].invocation.module_args.device_name

  • debug: var=ec2_snap.results[0].item

  • debug: var=ec2_snap.results[0].volume_id

  • debug: var=ec2_snap.results[0].snapshot_id

  • name: “Task 2: Create files for each instance”
    command: touch /home/ansible/playbooks/common/variables/‘{{ item }}.yml’
    with_items: “{{ awsec2id }}”

  • name: “Task 3: Add string instance_id to file for each instance”
    shell: ‘printf “{{ item.volume_id }}” > /home/ansible/playbooks/common/variables/“{{ item.item }}”’
    with_items: “{{ ec2_snap.results }}”

Error making code:

  • name: “Task 3: Add string instance_id to file for each instance”
    shell: ‘printf “{{ item.volume_id }}” > /home/ansible/playbooks/common/variables/“{{ item.item }}”’
    with_items: “{{ ec2_snap.results }}”

ec2_snap registered variable is having details of each of two snapshots in a relevant array element. How to create loop on them?

Short Output is as follow:

{
“changed”:false,
“ec2_snap”:{
“changed”:true,
“msg”:“All items completed”,
“results”:[
{ },
{ }
]
}
}

The extended output is as follow:

ok: [localhost] => {
“changed”: false,
“ec2_snap”: {
“changed”: true,
“msg”: “All items completed”,
“results”: [
{
“_ansible_item_result”: true,
“_ansible_no_log”: false,
“_ansible_parsed”: true,
“changed”: true,
“invocation”: {
“module_args”: {
“aws_access_key”: null,
“aws_secret_key”: null,
“description”: “snapshot of /dev/sda1 from i-037b5960651XXXXX taken 2017/03/22-17:19”,
“device_name”: “/dev/sda1”,
“ec2_url”: null,
“instance_id”: “i-037b59606518XXXXX”,
“last_snapshot_min_age”: 0,
“profile”: null,
“region”: “us-east-1”,
“security_token”: null,
“snapshot_id”: null,
“snapshot_tags”: {
“frequency”: “daily”,
“source”: “/dev/sda1”
},
“state”: “present”,
“validate_certs”: true,
“volume_id”: null,
“wait”: true,
“wait_timeout”: 0
}
},
“item”: “i-037b59606518XXXXX”,
“snapshot_id”: “snap-08d7aeda75dbXXXXX”,
“tags”: {
“frequency”: “daily”,
“source”: “/dev/sda1”
},
“volume_id”: “vol-09aec429dfcfXXXXX”,
“volume_size”: 8
},
{
“_ansible_item_result”: true,
“_ansible_no_log”: false,
“_ansible_parsed”: true,
“changed”: true,
“invocation”: {
“module_args”: {
“aws_access_key”: null,
“aws_secret_key”: null,
“description”: “snapshot of /dev/sda1 from i-04452f010e4XXXXX taken 2017/03/22-17:20”,
“device_name”: “/dev/sda1”,
“ec2_url”: null,
“instance_id”: “i-04452f010e4cXXXXX”,
“last_snapshot_min_age”: 0,
“profile”: null,
“region”: “us-east-1”,
“security_token”: null,
“snapshot_id”: null,
“snapshot_tags”: {
“frequency”: “daily”,
“source”: “/dev/sda1”
},
“state”: “present”,
“validate_certs”: true,
“volume_id”: null,
“wait”: true,
“wait_timeout”: 0
}
},
“item”: “i-04452f010e4caXXXXX”,
“snapshot_id”: “snap-0f8cb9b9d7b4XXXXX”,
“tags”: {
“frequency”: “daily”,
“source”: “/dev/sda1”
},
“volume_id”: “vol-0e0516ea9623XXXXX”,
“volume_size”: 15
}
]
}
}

  • Can it be resolved with with_submemebrs?
  • Can someone please help me to understand?

<snip>

I this output of ec2_snap is correct there is nothing wrong with your loop code.

Since you haven't provided any output of the error it's impossible to help.

Piece of code is as follow:

# Manual accessing sub-elements

  • debug: var=ec2_snap
  • debug: var=ec2_snap.results[0].invocation.module_args.device_name
  • debug: var=ec2_snap.results[0].item
  • debug: var=ec2_snap.results[0].volume_id
  • debug: var=ec2_snap.results[0].snapshot_id
  • debug: var=ec2_snap.results[1].invocation.module_args.device_name
  • debug: var=ec2_snap.results[1].item
  • debug: var=ec2_snap.results[1].volume_id
  • debug: var=ec2_snap.results[1].snapshot_id

# Works fine, Output for above code is as follow:

ok: [localhost] => {
“changed”: false,
“ec2_snap.results[0].invocation.module_args.device_name”: “/dev/sda1”
}

TASK [debug] ***********************************************************************************************************************************************************
task path: /home/ansible/playbooks/rol1.yml:27
ok: [localhost] => {
“changed”: false,
“ec2_snap.results[0].item”: “i-037b59606518XXXXX”
}

TASK [debug] ***********************************************************************************************************************************************************
task path: /home/ansible/playbooks/rol1.yml:28
ok: [localhost] => {
“changed”: false,
“ec2_snap.results[0].volume_id”: “vol-09aec429dfcXXXXX”
}

TASK [debug] ***********************************************************************************************************************************************************
task path: /home/ansible/playbooks/rol1.yml:29
ok: [localhost] => {
“changed”: false,
“ec2_snap.results[0].snapshot_id”: “snap-0fe95f5d794XXXXX”
}

TASK [debug] ***********************************************************************************************************************************************************
task path: /home/ansible/playbooks/rol1.yml:30
ok: [localhost] => {
“changed”: false,
“ec2_snap.results[1].invocation.module_args.device_name”: “/dev/sda1”
}

TASK [debug] ***********************************************************************************************************************************************************
task path: /home/ansible/playbooks/rol1.yml:31
ok: [localhost] => {
“changed”: false,
“ec2_snap.results[1].item”: “i-04452f010e4cXXXXX”
}

TASK [debug] ***********************************************************************************************************************************************************
task path: /home/ansible/playbooks/rol1.yml:32
ok: [localhost] => {
“changed”: false,
“ec2_snap.results[1].volume_id”: “vol-0e0516ea962XXXXX”
}

TASK [debug] ***********************************************************************************************************************************************************
task path: /home/ansible/playbooks/rol1.yml:33
ok: [localhost] => {
“changed”: false,
“ec2_snap.results[1].snapshot_id”: “snap-0fa1ec99d379XXXXX”
}

When trying to access with_items, with following code:

  • name: “Task 3: Add string instance_id to file for each instance”
    shell: ‘printf “{{ item.volume_id }}”’
    #shell: ‘printf “{{ item.volume_id }}” > /home/ansible/playbooks/common/variables/“{{ item.item }}”’
    with_items:
  • “{{ ec2_snap.results }}”

Rather than giving proper output, as appeared in above manual access, it gives garbrage as follow:

changed: [localhost] => (item={‘_ansible_parsed’: True, u’changed’: True, ‘_ansible_no_log’: False, ‘item’: u’i-04452f010e4cXXXXX’, ‘_ansible_item_result’: True, u’volume_size’: 15, u’snapshot_id’: u’snap-0fa1ec99d379XXXXX’, u’volume_id’: u’vol-0e0516ea9623XXXXX’, u’invocation’: {u’module_args’: {u’aws_secret_key’: None, u’profile’: None, u’aws_access_key’: None, u’description’: u’snapshot of /dev/sda1 from i-04452f010e4cXXXXX taken 2017/03/22-19:58’, u’security_token’: None, u’snapshot_tags’: {u’source’: u’/dev/sda1’, u’frequency’: u’daily’}, u’region’: u’us-east-1’, u’snapshot_id’: None, u’device_name’: u’/dev/sda1’, u’instance_id’: u’i-04452f010e4cXXXXX’, u’state’: u’present’, u’wait_timeout’: 0, u’ec2_url’: None, u’last_snapshot_min_age’: 0, u’validate_certs’: True, u’volume_id’: None, u’wait’: True}}, u’tags’: {u’source’: u’/dev/sda1’, u’frequency’: u’daily’}}) => {
“changed”: true,
“cmd”: “printf "vol-0e0516ea9623XXXXX"”,
“delta”: “0:00:00.002131”,
“end”: “2017-03-22 19:58:13.225366”,
“invocation”: {
“module_args”: {
“_raw_params”: “printf "vol-0e0516ea962XXXXX"”,
“_uses_shell”: true,
“chdir”: null,
“creates”: null,
“executable”: null,
“removes”: null,
“warn”: true
}
},
“item”: {
“changed”: true,
“invocation”: {
“module_args”: {
“aws_access_key”: null,
“aws_secret_key”: null,
“description”: “snapshot of /dev/sda1 from i-04452f010e4cacb0b taken 2017/03/22-19:58”,
“device_name”: “/dev/sda1”,
“ec2_url”: null,
“instance_id”: “i-04452f010e4cXXXXX”,
“last_snapshot_min_age”: 0,
“profile”: null,
“region”: “us-east-1”,
“security_token”: null,
“snapshot_id”: null,
“snapshot_tags”: {
“frequency”: “daily”,
“source”: “/dev/sda1”
},
“state”: “present”,
“validate_certs”: true,
“volume_id”: null,
“wait”: true,
“wait_timeout”: 0
}
},
“item”: “i-04452f010e4XXXXX”,
“snapshot_id”: “snap-0fa1ec99d379XXXXX”,
“tags”: {
“frequency”: “daily”,
“source”: “/dev/sda1”
},
“volume_id”: “vol-0e0516ea9623XXXXX”,
“volume_size”: 15
},
“rc”: 0,
“start”: “2017-03-22 19:58:13.223235”,
“stderr”: “”,
“stderr_lines”: ,
“stdout”: “vol-0e0516ea962XXXXX”,
“stdout_lines”: [
“vol-0e0516ea9623XXXXX”
]
}

Same way another looping issue is as follow (actual code):

  • name: “Task 2: Fetch EC2 details”
    ec2_remote_facts:
    aws_access_key: ‘{{ AwsAccesskey }}’
    aws_secret_key: ‘{{ AwsSecretkey }}’
    region: ‘{{ region }}’
    filters:
    instance_id: ‘{{ item }}’
    register: ec2_hosts
    with_items: “{{ awsec2id }}”
  • debug: var=ec2_hosts
  • debug: var=ec2_hosts.results[0].instances[0].private_ip_address
  • debug: var=ec2_hosts.results[1].instances[0].private_ip_address

with_items: Looping for the above:

  • name: looping
    shell: ‘printf “{{ item.instance[0].private_ip_address }}”’
    with_items: “{{ ec2_hosts.results }}”

Error:
TASK [awsAtachEBSvolume : looping] *************************************************************************************************************************************
fatal: [localhost]: FAILED! => {“failed”: true, “msg”: “the field ‘args’ has an invalid value, which appears to include a variable that is undefined. The error was: ‘dict object’ has no attribute ‘instance’\n\nThe error appears to have been in ‘/home/ansible/playbooks/roles/awsAtachEBSvolume/tasks/main.yml’: line 30, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n - debug: var=awsec2id\n - name: looping\n ^ here\n”}
to retry, use: --limit @/home/ansible/playbooks/rol.retry

Output registered variable ec2_hosts, follow:
ok: [localhost] => {
“changed”: false,
“ec2_hosts”: {
“changed”: false,
“msg”: “All items completed”,
“results”: [
{
“_ansible_item_result”: true,
“_ansible_no_log”: false,
“_ansible_parsed”: true,
“changed”: false,
“instances”: [
{
“ami_launch_index”: “0”,
“architecture”: “x86_64”,
“block_device_mapping”: [
{
“attach_time”: “2017-03-17T08:02:13.000Z”,
“delete_on_termination”: false,
“device_name”: “/dev/xvda”,
“status”: “attached”,
“volume_id”: “vol-0573516d4320XXXXX”
},
{
“attach_time”: “2017-03-21T08:32:27.000Z”,
“delete_on_termination”: false,
“device_name”: “/dev/xvdl”,
“status”: “attached”,
“volume_id”: “vol-07556df02e90XXXXX”
}
],
“client_token”: “IQNIH14887952XXXXX”,
“ebs_optimized”: false,
“groups”: [
{
“id”: “sg-dbcXXXXX”,
“name”: “launch-wizard-14”
}
],
“hypervisor”: “xen”,
“id”: “i-0003df8d26bXXXXX”,
“image_id”: “ami-0b33XXXXX”,
“instance_profile”: null,
“interfaces”: [
{
“id”: “eni-8fdfbXXXXX”,
“mac_address”: “0e:df:73:4c:67:e6”
}
],
“kernel”: null,
“key_name”: “XXXXX”,
“launch_time”: “2017-03-21T04:02:10.000Z”,
“monitoring_state”: “disabled”,
“persistent”: false,
“placement”: {
“tenancy”: “default”,
“zone”: “us-east-1c”
},
“private_dns_name”: “ip-XXXXX.ec2.internal”,
“private_ip_address”: “172.XXXXX”,
“public_dns_name”: “ec2-XXXXX-156.compute-1.amazonaws.com”,
“public_ip_address”: “54.89.XXXXX”,
“ramdisk”: null,
“region”: “us-east-1”,
“requester_id”: null,
“root_device_type”: “ebs”,
“source_destination_check”: “true”,
“spot_instance_request_id”: null,
“state”: “running”,
“tags”: {
“Name”: “XXXXX”
},
“virtualization_type”: “hvm”,
“vpc_id”: “vpc-76XXXXX”
}
],
“invocation”: {
“module_args”: {
“aws_access_key”: “AKIAIGKMK7EXXXXX”,
“aws_secret_key”: “VALUE_SPECIFIED_IN_NOXXXXX”,
“ec2_url”: null,
“filters”: {
“instance_id”: “i-0003df8d26XXXXX”
},
“profile”: null,
“region”: “us-east-1”,
“security_token”: null,
“validate_certs”: true
}
},
“item”: “i-0003df8d26b5XXXXX”
},
{
“_ansible_item_result”: true,
“_ansible_no_log”: false,
“_ansible_parsed”: true,
“changed”: false,
“instances”: [
{
“ami_launch_index”: “0”,
“architecture”: “x86_64”,
“block_device_mapping”: [
{
“attach_time”: “2017-03-21T08:32:38.000Z”,
“delete_on_termination”: false,
“device_name”: “/dev/xvdl”,
“status”: “attached”,
“volume_id”: “vol-0118435137XXXXX”
},
{
“attach_time”: “2017-03-22T08:28:47.000Z”,
“delete_on_termination”: false,
“device_name”: “/dev/sda1”,
“status”: “attached”,
“volume_id”: “vol-0e0516ea96XXXXX”
}
],
“client_token”: “CrBZB1489737XXXXX”,
“ebs_optimized”: false,
“groups”: [
{
“id”: “sg-13d8076c”,
“name”: “launch-wizard-19”
}
],
“hypervisor”: “xen”,
“id”: “i-04452f010e4cXXXXX”,
“image_id”: “ami-2051XXXXX”,
“instance_profile”: null,
“interfaces”: [
{
“id”: “eni-4c0efXXXXX”,
“mac_address”: “0e:d9:48:XXXXX”
}
],
“kernel”: null,
“key_name”: “bparmaXXXXX”,
“launch_time”: “2017-03-22T10:23:26.000Z”,
“monitoring_state”: “disabled”,
“persistent”: false,
“placement”: {
“tenancy”: “default”,
“zone”: “us-east-1c”
},
“private_dns_name”: “ip-172-31-XXXXX.ec2.internal”,
“private_ip_address”: “172.31XXXXX”,
“public_dns_name”: “ec2-34-207-XXXXX.compute-1.amazonaws.com”,
“public_ip_address”: “34.207.XXXXX”,
“ramdisk”: null,
“region”: “us-east-1”,
“requester_id”: null,
“root_device_type”: “ebs”,
“source_destination_check”: “true”,
“spot_instance_request_id”: null,
“state”: “running”,
“tags”: {
“Name”: “XXXXX”
},
“virtualization_type”: “hvm”,
“vpc_id”: “vpc-76b6XXXXX”
}
],
“invocation”: {
“module_args”: {
“aws_access_key”: “AKIAIGKMK7E3O2RXXXXX”,
“aws_secret_key”: “VALUE_SPECIFIED_IN_NO_XXXXX”,
“ec2_url”: null,
“filters”: {
“instance_id”: “i-04452f010XXXXX”
},
“profile”: null,
“region”: “us-east-1”,
“security_token”: null,
“validate_certs”: true
}
},
“item”: “i-04452f010e4cXXXXX”
}
]
}
}

*When trying to access with_items, with following code:*

   - name: "Task 3: Add string instance_id to file for each instance"
     shell: 'printf "{{ item.volume_id }}"'
     #shell: 'printf "{{ item.volume_id }}" >
/home/ansible/playbooks/common/variables/"{{ item.item }}"'
     with_items:
       - "{{ ec2_snap.results }}"

*Rather than giving proper output, as appeared in above manual access, it
gives garbrage as follow:*

This is not garbage, its a verbose output of what Ansible is doing.
I have commented the interesting part below.

changed: [localhost] => (item={'_ansible_parsed': True, u'changed': True,
'_ansible_no_log': False, 'item': u'i-04452f010e4cXXXXX',
'_ansible_item_result': True, u'volume_size': 15, u'snapshot_id':
u'snap-0fa1ec99d379XXXXX', u'volume_id': u'vol-0e0516ea9623XXXXX',
u'invocation': {u'module_args': {u'aws_secret_key': None, u'profile': None,
u'aws_access_key': None, u'description': u'snapshot of /dev/sda1 from
i-04452f010e4cXXXXX taken 2017/03/22-19:58', u'security_token': None,
u'snapshot_tags': {u'source': u'/dev/sda1', u'frequency': u'daily'},
u'region': u'us-east-1', u'snapshot_id': None, u'device_name':
u'/dev/sda1', u'instance_id': u'i-04452f010e4cXXXXX', u'state': u'present',
u'wait_timeout': 0, u'ec2_url': None, u'last_snapshot_min_age': 0,
u'validate_certs': True, u'volume_id': None, u'wait': True}}, u'tags':
{u'source': u'/dev/sda1', u'frequency': u'daily'}}) => {
    "changed": true,
    "cmd": "printf \"vol-0e0516ea9623XXXXX\"",

Here you see that ansible is running
printf "vol-0e0516ea9623XXXXX"
as you wanted with you loop.

    "delta": "0:00:00.002131",
    "end": "2017-03-22 19:58:13.225366",
    "invocation": {
        "module_args": {
            "_raw_params": "printf \"vol-0e0516ea962XXXXX\"",
            "_uses_shell": true,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "warn": true
        }
    },
    "item": {
        "changed": true,
        "invocation": {
            "module_args": {
                "aws_access_key": null,
                "aws_secret_key": null,
                "description": "snapshot of /dev/sda1 from
i-04452f010e4cacb0b taken 2017/03/22-19:58",
                "device_name": "/dev/sda1",
                "ec2_url": null,
                "instance_id": "i-04452f010e4cXXXXX",
                "last_snapshot_min_age": 0,
                "profile": null,
                "region": "us-east-1",
                "security_token": null,
                "snapshot_id": null,
                "snapshot_tags": {
                    "frequency": "daily",
                    "source": "/dev/sda1"
                },
                "state": "present",
                "validate_certs": true,
                "volume_id": null,
                "wait": true,
                "wait_timeout": 0
            }
        },
        "item": "i-04452f010e4XXXXX",
        "snapshot_id": "snap-0fa1ec99d379XXXXX",
        "tags": {
            "frequency": "daily",
            "source": "/dev/sda1"
        },
        "volume_id": "vol-0e0516ea9623XXXXX",
        "volume_size": 15
    },
    "rc": 0,
    "start": "2017-03-22 19:58:13.223235",
    "stderr": "",
    "stderr_lines": ,
    "stdout": "vol-0e0516ea962XXXXX",
    "stdout_lines": [
        "vol-0e0516ea9623XXXXX"

The output of the command above is in stdout and stdout_lines.
vol-0e0516ea962XXXXX is the output of the printf command.

This is working as it should, what were you expecting?

*Same way another looping issue is as follow (actual code):*

It's no issue here, it's working as designed.
To understand, we need to now what you would expect.

  - name: "Task 2: Fetch EC2 details"
    ec2_remote_facts:
      aws_access_key: '{{ AwsAccesskey }}'
      aws_secret_key: '{{ AwsSecretkey }}'
      region: '{{ region }}'
      filters:
        instance_id: '{{ item }}'
    register: ec2_hosts
    with_items: "{{ awsec2id }}"

The output of is registered in variable ec2_hosts

  - debug: var=ec2_hosts

And the output is a list, one for each item, in ec2_hosts.results.
So the output below is as expected.

The error messages says "dict object' has no attribute 'instance'"
If you check the output of ec2_hosts you will see it's called
instances and not instance (missing a "s" at the end)

Hey Kai,

Thanks for your guidance. I am just looking for vol-0e0516ea962XXXXX as output. Same way I have to grab various parts from output, to convert them in key : value pair, to store in YAML file.

Please, can you suggest how to only get the particular part? Like we can get with following manual way?

  • debug: var=ec2_snap.results[0].invocation.module_args.device_name

  • debug: var=ec2_snap.results[0].item

  • debug: var=ec2_snap.results[0].volume_id

  • debug: var=ec2_snap.results[0].snapshot_id

  • debug: var=ec2_snap.results[1].invocation.module_args.device_name

  • debug: var=ec2_snap.results[1].item

  • debug: var=ec2_snap.results[1].volume_id

  • debug: var=ec2_snap.results[1].snapshot_id

I'm not sure what you mean, because your loop did that.
The output on the screen will always contain more than just the value, but the variable itself contains just the value.

- name: Print out variables in a loop with debug module
   debug: msg="### The output ###: {{ item.invocation.module_args.device_name }} - {{ item.item }} - {{ item.volume_id }} - {{ item.snapshot_id }}"
   with_items: "{{ ec2_snap.results }}"

This will print the values on one line divided by dash.
And it would be easy to identify since the line start with "### The output ###:"

Kai,

Thanks for your all support. :slight_smile:

You were right it was just displaying access details on the screen. When I started putting them in a YAML file, I was just getting what I needed.

I have created three Ansible Roles and uploaded them to https://github.com/Bhavin-P/Ansible
Role 1. To add, format and mount extra EBS volume to the given list of EC2 IDs.
Role 2. To take a snapshot of root volumes, it can be scheduled cron.
Role 3. To restore root volume, from the snapshot taken by Role 2.

Thanks to you and community as well, for providing the platform.

Thanks and regards,
Bhavin Parmar.