next branch: question about boto3_conn (from lib/ansible/module_utils/ec2.py) and STS

Archives Ansible Developer
1

Hi,

I’m currently writing an EMR module for ansible at $WORK (sorry no plan for now to open source it) and I’m trying to use boto3, and I have to admit I’m confused with logic behind boto3_conn().
I have to use a role to access our environment so I must use sts_assume_role to gain privileges. Unfortunately, my “security_token” is set to None since rev 27398131cf31eb7ca834a30ea2d8a871a937a377 (https://github.com/ansible/ansible/commit/27398131cf31eb7ca834a30ea2d8a871a937a377)
Here’s the non working pseudo-code (copy’n’paste from ECS ones)

class EMRClusterManager:
“”" Handles EMR clusters"“”
def init(self, module):
self.module = module
try:
region, ec2_url, aws_connect_kwargs = get_aws_connection_info(module, boto3=True)
if not region:
module.fail_json(msg=“Region must be specified as a parameter, in EC2_REGION or AWS_REGION environment variables or in boto configuration file”)
self.emr = boto3_conn(module, conn_type=‘client’, resource=‘emr’, region=region, endpoint=ec2_url, **aws_connect_kwargs)

XXX doesn’t catch error IRL

except boto.exception.NoAuthHandlerFound, e:
self.module.fail_json(msg="Can’t authorize connection - "+str(e))

def main()

argument_spec = ec2_argument_spec()

argument_spec.update( [SNIP] )

module = AnsibleModule(argument_spec=argument_spec)

emr = EMRClusterManager(module)

[SNIP]

it fails with : “The security token included in the request is invalid.” error message.

IMHO, the culpit is here (https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/ec2.py#L45)

def boto3_conn(module, conn_type=None, resource=None, region=None, endpoint=None, **params):
profile = params.pop(‘profile_name’, None)
params[‘aws_session_token’] = params.pop(‘security_token’, None)

params[‘aws_session_token’] is already set via get_aws_connection_info() line 151

if HAS_BOTO3 and boto3:
boto_params = dict(aws_access_key_id=access_key,
aws_secret_access_key=secret_key,
aws_session_token=security_token)
if validate_certs:
boto_params[‘verify’] = validate_certs

if profile_name:
boto_params[‘profile_name’] = profile_name

removing the last line fix the problem (params[‘aws_session_token’] = params.pop(‘security_token’, None))

Since I’m not very familiar with ansible core, I wonder if I missed something.

Thanks for your help

clem

2

Hmm so not sure why it’s not working for you. However, to explain:

Lines 2 & 3 in boto3_conn() re label the fields of the param dict b/c, as you mentioned it brings the dict over form get_aws_connection_info(), those two entries for the temp token and cert validation have different keywords in boto vs boto3.

Could you elaborate a bit more on how you are using sts_assume_role? Perhaps a sanitized version of the tasks relevant to this issue?

In the mean time, I’m going to try and replicate with other modules.

3

Hi,

Since we use several roles to make some sanity checks, add tags, etc. I post you the simpliest version. Note that it works for our ec2/s3/rds roles. we don’t use route53 or ecs module so I only can presume their behavior with boto3

  • name: get token
    sts_assume_role:
    role_arn: “{{ ValidARN[role].arn }}”
    role_session_name: “{{ ValidARN[role].session_name }}”
    region: “{{ region }}”
    register: assumed_role

  • name: create EMR cluster
    ec2_emr:
    aws_access_key: “{{ assumed_role.sts_creds.access_key }}”
    aws_secret_key: “{{ assumed_role.sts_creds.secret_key }}”
    security_token: “{{ assumed_role.sts_creds.session_token }}”
    key_name: “{{ key }}”
    region: “{{ region }}”
    name: test ansible
    state: present
    emr_version: emr-4.0.0

[…]

if I worked on python script generated (with ANSIBLE_KEEP_REMOTE_FILES defined)
here are the loosely obfuscated module args:

{“aws_secret_key”: “XXXXXX”, “aws_access_key”: “YYYYYYY”, “name”: “test ansible”, “security_token”: “ZZZZZZ”, “tags”: {“my_tag”: “foo”}, “key_name”: “key_bar”, “region”: “eu-west-1”, “instances”: [{“InstanceRole”: “MASTER”, “InstanceCount”: 1, “Name”: “master”, “Market”: “ON_DEMAND”, “InstanceType”: “m3.xlarge”}, {“InstanceRole”: “CORE”, “InstanceCount”: 2, “Name”: “master”, “Market”: “ON_DEMAND”, “InstanceType”: “m3.xlarge”}, {“InstanceRole”: “TASK”, “InstanceCount”: 0, “Name”: “master”, “Market”: “ON_DEMAND”, “InstanceType”: “m3.xlarge”}], “state”: “present”, “vpc_id”: “vpc-XXXXX”, “emr_version”: “emr-4.0.0”, “applications”: [{“Version”: “2.6.0”, “Name”: “Hadoop”}, {“Version”: “3.7.1”, “Name”: “Hue”}, {“Version”: “0.14.0”, “Name”: “Hive”}]}’

if I modify boto3_conn inside that script

def boto3_conn(module, conn_type=None, resource=None, region=None, endpoint=None, **params):
profile = params.pop(‘profile_name’, None)
print "PRE TOKEN BOTO3_CONN "
print params[‘aws_session_token’]
params[‘aws_session_token’] = params.pop(‘security_token’, None)
print "POST TOKEN BOTO3_CONN "

the output is :

PRE TOKEN BOTO3_CONN
ZZZZZZ
POST TOKEN BOTO3_CONN
None

boto3_conn clearly overrides valid parameters.

clem

4

I opened a pull request:
https://github.com/ansible/ansible/pull/13283