Working around lookup plugin limitation with environment variables

Rob_White · April 10, 2017, 7:40am

Hi all,

I’m looking for any suggestions on how to work around the following issue:

I am trying to use a lookup plugin but it is ignoring my environment variables.

The lookup plugin I am trying to use is credstash and the environment variables are being set because I am using AWS STS to assume a role.

It looks a little like this:

`

---
- name: Get STS token
  hosts: "{{ hosts }}"
  connection: local
  tasks:
    - name: Assume role for passed account
      sts_assume_role:
        region: "{{ aws_region }}"
        role_arn: "arn:aws:iam::{{ aws_account_id }}:role/ansible-is_assumed"
        role_session_name: ansible_assumed_role
      register: assumed_role
      when: use_sts

    - name: Set environment with AWS creds when available
      set_fact:
        my_environment:
          AWS_ACCESS_KEY_ID: "{{ assumed_role.sts_creds.access_key }}"
          AWS_SECRET_ACCESS_KEY: "{{ assumed_role.sts_creds.secret_key }}"
          AWS_SECURITY_TOKEN: "{{ assumed_role.sts_creds.session_token }}"
      when: assumed_role.skipped is not defined

    - name: Set environment blank when no AWS creds
      set_fact:
        my_environment: {}
      when: assumed_role.skipped is defined

- name: Deploy infrastructure role
  hosts: "{{ hosts }}"
  connection: local
  roles:
    - "{{ role_to_deploy }}"
  environment: "{{ my_environment }}"

`

Now in the role_to_deploy i have lots of tasks that work brilliantly calling out to AWS using the STS credentials supplied as intended.

Unfortunately, any lookup plugins do not work properly because they do not use the environment variables I have set. They end up calling out to AWS using the default credentials rather than being overridden by the STS creds.

I don’t think this is a bug but it is never the less annoying.

I’m hoping maybe one of the core team can think of a way I can get the lookup plugin to ‘see’ my STS environment variables I have set.

Brian_Coca · April 10, 2017, 5:07pm

The environment keyword ONLY affects remote task execution (invoked
inside connection), not the 'local' environment which is what lookups
run under.

Rob_White · April 11, 2017, 12:18am

Indeed, hence why it doesn’t work.

Is there any other way to pass an environment variable to a lookup plugin?

The only thing I’ve come up with is a not very elegant:

lookup('pipe', '(export AWS_ACCESS_KEY_ID=' + assumed_role.sts_creds.access_key + ' && export AWS_SECRET_ACCESS_KEY=' + assumed_role.sts_creds.secret_key + ' && export AWS_SECURITY_TOKEN=' + assumed_role.sts_creds.session_token + ' && credstash -r ' + aws_region + ' -t ' + env + '-credstash get rds_instance_mysql-1_username)')

This then fails when I am not using STS.

Brian_Coca · April 11, 2017, 12:59pm

set them in the environment before executing Ansible

AWS_SECURITY_TOKEN=xxx ansible-playbook ....

Rob_White · April 12, 2017, 1:41am

The problem is it is the ansible-playbook that is actually going off to get the STS creds using sts_assume_role module so until i run the play, i don’t have the creds.

I guess I could invoke a playbook from within another playbook

Nic_H · July 16, 2020, 10:05pm

Hey, have you ever find a solution here ?
I face the same problem trying to pass my sts credentials in environment for a role, like this :

pre_tasks :

name: Assume role
sts_assume_role:
role_arn: “arn:aws:iam:::role/myrole”
role_session_name: “mysession”
region: “{{ my_region }}”
register: assumed_role

roles:

myrole

environment:
AWS_ACCESS_KEY_ID: “{{ assumed_role.sts_creds.access_key }}”
AWS_SECRET_ACCESS_KEY: “{{ assumed_role.sts_creds.secret_key }}”
AWS_SECURITY_TOKEN: “{{ assumed_role.sts_creds.session_token }}”

however it doesn’t work because environment is evaluated before everything else, so my pre-task has not been run yet and the credentials (assumed_role variable) doesn’t exist .
not sure how to pass credential as environment in this case ?

racke · July 17, 2020, 12:15pm

Hello Nico,

I recommended to switch to tasks only in your playbook instead of using pre_tasks and roles keywords.
This should solve your problem and in my opinion it also looks more consistent:

tasks:
  - name: Assume role
    sts_assume_role:
      role_arn: "arn:aws:iam:::role/myrole"
      role_session_name: "mysession"
      region: "{{ my_region }}"
    register: assumed_role
  - name: Import my role
    import_role:
      name: myrole
    environment:
      AWS_ACCESS_KEY_ID: "{{ assumed_role.sts_creds.access_key }}"
      AWS_SECRET_ACCESS_KEY: "{{ assumed_role.sts_creds.secret_key }}"
      AWS_SECURITY_TOKEN: "{{ assumed_role.sts_creds.session_token }}"

Regards
Racke

Nic_H · July 17, 2020, 6:02pm

Thanks Racke,
sure , so in this case environment is assigned to the tasks level and should therefore not fail cause it is not loaded before runtime.
In the case I have multiple roles, I have to repeat the task as many times as I need a role , lots of duplicate environment then .

Can we pass a list or role in import_role or include_role ? I couldn’t make this working with “loop”

cheers


- include_role:
        - name: myrole1, when: something | default(None) 
        - name: myrole2, when: somethingElse | default(None)
  environment:
      AWS_ACCESS_KEY_ID: "{{ assumed_role.sts_creds.access_key }}"
      AWS_SECRET_ACCESS_KEY: "{{ assumed_role.sts_creds.secret_key }}"
      AWS_SECURITY_TOKEN: "{{ assumed_role.sts_creds.session_token }}"

racke · July 17, 2020, 6:12pm

Thanks Racke,
sure , so in this case */environment /*is assigned to the /*tasks */level and should therefore not fail cause it is not
loaded before runtime.
In the case I have multiple roles, I have to repeat the task as many times as I need a role , lots of
duplicate /*environment */then .

Can we pass a list or role in /*import_role */or /*include_role */? I couldn't make this working with "/*loop*/"

cheers

Hello Nico,

Loop is the wrong approach. But you can certainly set up a variable with your desired environment:

- name: My env
  set_fact:
    myenv:
      AWS_ACCESS_KEY_ID: "{{ assumed_role.sts_creds.access_key }}"
      AWS_SECRET_ACCESS_KEY: "{{ assumed_role.sts_creds.secret_key }}"
      AWS_SECURITY_TOKEN: "{{ assumed_role.sts_creds.session_token }}"

And in your import_role/include_role:

  - name: Import my role
    import_role:
      name: myrole
    environment: "{{ myenv }}"

Regards
Racke

Nic_H · July 23, 2020, 2:59am

Thank you for the proposition, I ve been trying for days to put this in every direction I could but unfortunately 'environment ’
was never passed to ‘import_role’ .
I am not sure how to investigate about import_role module and why environment is not supported , seems impossible to reuse sts.creds then .
Any idea is welcome

Dick_Visser · July 23, 2020, 4:27am

Your setting it in one play and then try to use it in another. That won’t work.

Other question. You run this sts task for all hosts, but is that needed for all of them?

Nic_H · July 24, 2020, 5:44pm

not sure if that needs new topic but I see that other AWS modules also fail to pickup variables in Environment as they should .

Example is rds_instance_facts

again, I set up the sts credentials in environment block, and then the module doesn’t use them ,why is that ?

`
Saisissez le code ici…

playbook.yml

pre_tasks:
`

name: Assume teardown devel stack role
sts_assume_role:
role_arn: “arn:aws:iam:::role/role”
role_session_name: “session”
region: “{{ region }}”
register: assumed_role

no_log: false

name: import roles
block:
import_role:
name: test2
when: cf_region is defined
environment:
aws_access_key: “{{ assumed_role.sts_creds.access_key }}”
aws_secret_key: “{{ assumed_role.sts_creds.secret_key }}”
security_token: "{{ assumed_role.sts_creds.session_token }}

test2.yml

name: Get rds facts
rds_instance_facts:

command: facts

db_instance_identifier: “{{ cf_stack_name }}”
region: “{{ cf_region }}”
register: rds

in the exec I see the temporary keys:

EXEC /bin/sh -c 'aws_secret_key=cHAyHECUKdRXeMNELOADIjPKRwdYfJR/BIY8nVMZ aws_access_key=ASIA3K27TU7xxxxxxxxxxxx
security_token=FwoGZXIvYXdzEOv//////////wEaDBTRsHoNaa4W+L0bkiK+ASTwafNPh9h+q2E6IqgZAR/MjX4eDpo/Vh2zLfFX+D/XHH/B1fCcLUaGp+6AXc0H/wqJG58dt9SQP5Dym1iLq3Xzn3rVwmEc9U+0Q3PcdbczO4qwJS/A2mr1lnZ03HV+PsT0lFbActJJ65VHcNvVCqy4sSvRd+ykeBgkkiM6L35icm8eTr5RJdnTMb2tmcHogNXteaXbaag1AjABC/114coEMspTOCPjexiPtic3io67lQvIJPK9L3gqKsMMn4Ioq7js+AUyLer/WPTStS/PBgsU8WsLrlxd/iuaU4qGgquy4+6vY5H/X7vEaSMyT/8OKo6MVg==

but the module doesn’t care and bail with :
ClientError: An error occurred (AccessDenied) when calling the DescribeDBInstances operation: User: arn:aws:iam:::user/myuser is not authorized to perform: rds:DescribeDBInstances on resource

I am losing hope in ansible
What is the point of having a global environment if each and single AWS module requires a set of local credentials ?
please help me understand the logic here,
cheers

racke · July 24, 2020, 7:41pm

not sure if that needs new topic but I see that other AWS modules also fail to pickup variables in Environment as they
should .

You are sure these environment variables are lowercase? Usually they are uppercase, like AWS_ACCESS_KEY.

Ansible just passes them along as you see in your output.

Regards
Racke

system · July 27, 2020, 3:50pm

Another option, is to use 'default' filter:

  environment:
      AWS_ACCESS_KEY_ID: "{{ ars.access_key|default('') }}"
      AWS_SECRET_ACCESS_KEY: "{{ ars.secret_key|default('') }}"
      AWS_SECURITY_TOKEN: "{{ ars.session_token|default('') }}"
  vars:
       ars: '{{ assumed_role|default({}).sts_creds|default({}) }}'

Nic_H · July 28, 2020, 11:19pm

same problem, assumed_role is not defined yet, so ansible failed for Undefined variable, although I couldn’t get the syntax you provided to work for the ars var , is that the good syntax ?
I got :

An unhandled exception occurred while templating '{{assumed_role|default({}).sts_creds|default({})}}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: template error while templating string: expected token 'end of print statement', got '.'. String: {{assumed_role|default({}).sts_creds|default({})}}"

Nic_H · July 28, 2020, 11:31pm

not sure if that needs new topic but I see that other AWS modules also fail to pickup variables in Environment as they
should .

You are sure these environment variables are lowercase? Usually they are uppercase, like AWS_ACCESS_KEY.

Ansible just passes them along as you see in your output.

Regards
Racke

yes you r right, in uppercase it then works better with the "block / import_role / environment " structure .
Thank you

Topic		Replies	Views
Help with Amazon STS Module [sts_assume_role] Ansible Project fedora , rhel , aws	0	0	May 22, 2017
Credstash Plugin Lookup Failure with Variable Ansible Project	0	0	February 17, 2017
Lookup plugin returns failure Ansible Project	2	0	June 28, 2017
User environment vars in lookup plugin Ansible Project	1	2	September 13, 2018
Need a help to create S3 bucket using Ansible Ansible Project aws	28	0	September 2, 2019

Working around lookup plugin limitation with environment variables

playbook.yml

no_log: false

test2.yml

command: facts

Related topics