Ansible 2.x - AWS ec2_asg problem -> AttributeError: 'module' object has no attribute 'InvalidInstance'

Archives Ansible Project
1

I am running a playbook to update an existing AWS Autoscaling group and have started to encounter an error message that I was not seeing before.

Environment:

Amazon Linux

Python 2.7.10

ansible (2.0.0.2)

boto (2.38.0)

botocore (1.3.20)

running the playbook to update an existing autoscaling group, I started getting this error last week:

TASK [Update Autoscaling Group] ************************************************

`

task path: /opt/hephaestus/ansible/tasks/update-asg.yaml:13

ESTABLISH LOCAL CONNECTION FOR USER: root

localhost EXEC ( umask 22 && mkdir -p “$( echo $HOME/.ansible/tmp/ansible-tmp-1453161674.67-268535195031737 )” && echo “$( echo $HOME/.ansible/tmp/ansible-tmp-1453161674.67-268535195031737 )” )

localhost PUT /tmp/tmpSYEg2V TO /root/.ansible/tmp/ansible-tmp-1453161674.67-268535195031737/ec2_asg

localhost EXEC LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /root/.ansible/tmp/ansible-tmp-1453161674.67-268535195031737/ec2_asg; rm -rf “/root/.ansible/tmp/ansible-tmp-1453161674.67-268535195031737/” > /dev/null 2>&1

An exception occurred during task execution. The full traceback is:

Traceback (most recent call last):

File “/root/.ansible/tmp/ansible-tmp-1453161674.67-268535195031737/ec2_asg”, line 2998, in

main()

File “/root/.ansible/tmp/ansible-tmp-1453161674.67-268535195031737/ec2_asg”, line 2987, in main

create_changed, asg_properties=create_autoscaling_group(connection, module)

File “/root/.ansible/tmp/ansible-tmp-1453161674.67-268535195031737/ec2_asg”, line 2664, in create_autoscaling_group

wait_for_elb(connection, module, group_name)

File “/root/.ansible/tmp/ansible-tmp-1453161674.67-268535195031737/ec2_asg”, line 2526, in wait_for_elb

healthy_instances = elb_healthy(asg_connection, elb_connection, module, group_name)

File “/root/.ansible/tmp/ansible-tmp-1453161674.67-268535195031737/ec2_asg”, line 2500, in elb_healthy

except boto.exception.InvalidInstance, e:

AttributeError: ‘module’ object has no attribute ‘InvalidInstance’

fatal: [localhost]: FAILED! => {“changed”: false, “failed”: true, “invocation”: {“module_name”: “ec2_asg”}, “parsed”: false}

`

I have tried to change the version of boto and ansible, however the error now persists. I have not been able to identify any root cause or known bug other than the error message. I have tried switching a few options on and off in the playbook such as: wait_for_instances and lc_check.

Thanks!

2

Did you try on a vm (or in docker?) with old ansible 1.9.4? Just to prevent environmental “interference”.

3

I tried on multiple amazon linux hosts, both using native python 2.7.10 as well as a virtual environment for python 2.7.10. I have also tried with Ansible 1.9.4 and 2.0.0.2, and with boto 2.38.0 and 2.39.0. The same error persists:

`

failed: [localhost] => {“failed”: true, “parsed”: false}

Traceback (most recent call last):

File “/root/.ansible/tmp/ansible-tmp-1453831301.21-186005588822085/ec2_asg”, line 2455, in

main()

File “/root/.ansible/tmp/ansible-tmp-1453831301.21-186005588822085/ec2_asg”, line 2444, in main

create_changed, asg_properties=create_autoscaling_group(connection, module)

File “/root/.ansible/tmp/ansible-tmp-1453831301.21-186005588822085/ec2_asg”, line 2246, in create_autoscaling_group

wait_for_elb(connection, module, group_name)

File “/root/.ansible/tmp/ansible-tmp-1453831301.21-186005588822085/ec2_asg”, line 2113, in wait_for_elb

except boto.exception.InvalidInstance, e:

AttributeError: ‘module’ object has no attribute ‘InvalidInstance’

`

I can avoid the error if I do not use the option “wait_for_instances” in my Ansible config. So something in this code path is executing the call getting the error. The error itself is a Python/Boto error, as the task really does succeed (although Ansible does not know that).

In the file:

ansible/modules/core/cloud/amazon/ec2_asg.py

def elb_healthy(asg_connection, elb_connection, module, group_name):

`

healthy_instances =

as_group = asg_connection.get_all_groups(names=[group_name])[0]

props = get_properties(as_group)

get healthy, inservice instances from ASG

instances =

for instance, settings in props[‘instance_facts’].items():

if settings[‘lifecycle_state’] == ‘InService’ and settings[‘health_status’] == ‘Healthy’:

instances.append(instance)

log.debug(“ASG considers the following instances InService and Healthy: {0}”.format(instances))

log.debug(“ELB instance status:”)

for lb in as_group.load_balancers:

we catch a race condition that sometimes happens if the instance exists in the ASG

but has not yet show up in the ELB

try:

lb_instances = elb_connection.describe_instance_health(lb, instances=instances)

except boto.exception.InvalidInstance, e:

pass

for i in lb_instances:

if i.state == “InService”:

healthy_instances.append(i.instance_id)

log.debug(“{0}: {1}”.format(i.instance_id, i.state))

return len(healthy_instances)

`