# How to authorise an EC2 instance for S3 access

**URL:** <https://forum.ansible.com/t/how-to-authorise-an-ec2-instance-for-s3-access/18546>\
**Category:** Ansible Project\
**Tags:** aws\
**Created:** [June 19, 2015, 4:21am UTC](https://forum.ansible.com/t/how-to-authorise-an-ec2-instance-for-s3-access/18546 "2015-06-19T04:21:44Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Andrew\_Burrow](https://avatars.discourse-cdn.com/v4/letter/a/c5a1d2/32.png) [@Andrew\_Burrow](https://forum.ansible.com/u/Andrew_Burrow)\
**Post date:** [June 19, 2015, 4:21am UTC](https://forum.ansible.com/t/how-to-authorise-an-ec2-instance-for-s3-access/18546/1 "2015-06-19T04:21:44Z")

</div>

I am unable to make use of IAM roles in my Ansible playbooks. Specifically, I have authorised an EC2 instance to get from an S3 bucket, but I cannot work out how to make use of this authorisation from within Ansible.

**The question**

How do I write Ansible task(s) that satisfies all the following :

1. Runs on an EC2 instance
2. Uses the IAM role defined on the EC2 instance to obtain authorisation to access an S3 bucket
3. Gets a file from the S3 bucket

**A work around**

I can get the EC2 instance to download from S3, only by passing in my credentials as follows:

`

- name: Download the part archive from S3  
s3:  
aws\_access\_key: “{{ lookup(‘env’,‘aws\_key’) }}”  
aws\_secret\_key: “{{ lookup(‘env’,‘aws\_secret’) }}”  
region: “{{ aws\_packages\_region }}”  
bucket: “{{ aws\_packages\_bucket }}”  
object: “/JI79IML/my\_part\_X86\_64\_c7.15.tar.gz”  
dest: “/data/parts/JI79IML/my\_part\_X86\_64\_c7.15.tar.gz”  
mode: get  
overwrite: no

`

However, I would rather not send my AWS credentials to the instance. Instead I have defined a role with the appropriate permissions to get files from the S3 bucket.

**What I’ve tried**

The top answer in the stack overflow question linked below, suggests that it is simple matter of leaving the secret access key parameters out, and letting the Boto library take care of assuming the role.

- [http://stackoverflow.com/questions/28997757/ansible-and-s3-module](http://stackoverflow.com/questions/28997757/ansible-and-s3-module)

However, when I try this with Ansible 1.8.4 and Boto 2.36.0 I get

`  
msg: No handler was ready to authenticate. 1 handlers were checked. [‘HmacAuthV1Handler’] Check your credentials

`

and with Ansible 1.9.1 and Boto 2.38.0 I get:

`msg: Failed to connect to S3: 'module' object has no attribute 'connect_to_region'`

**How I’ve confirmed the IAM role**

To confirm that the IAM role is **_sufficient_** , I installed awscli on the EC2 instance and performed the download directly. First, I assumed the role

`

aws sts assume-role --role-arn “${ROLE\_ARN}” --role-session-name “GettingMyPart”

`

which returns an absolutely baffling error message that the user with the assumed role cannot assume the role?!? But seems to do the trick, because I can then download the part

`

aws s3api get-object --bucket “${BUCKET\_NAME}” --key JI79IML/my\_part\_X86\_64\_c7.15.tar.gz my\_part\_X86\_64\_c7.15.tar.gz

`

To confirm that the IAM role is _required_, I created another instance that does not enjoy a role and installed awscli on this second EC2 instance and followed the above steps. In each case, I got the message “Unable to locate credentials” as expected

---

<div class="post-metadata">

**Author:** ![benno\_joy](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@benno\_joy](https://forum.ansible.com/u/benno_joy)\
**Post date:** [June 19, 2015, 5:44am UTC](https://forum.ansible.com/t/how-to-authorise-an-ec2-instance-for-s3-access/18546/2 "2015-06-19T05:44:39Z")

</div>

Hi Andrew,

instance profiles do work without any issues, from the error msg: Failed to connect to S3: ‘module’ object has no attribute ‘connect\_to\_region’

seems like boto is not installed properly, how did you install boto ? can you please try reinstalling boto and check.

- Benno

---

<div class="post-metadata">

**Author:** ![Andrew\_Burrow](https://avatars.discourse-cdn.com/v4/letter/a/c5a1d2/32.png) [@Andrew\_Burrow](https://forum.ansible.com/u/Andrew_Burrow)\
**Post date:** [June 19, 2015, 6:52am UTC](https://forum.ansible.com/t/how-to-authorise-an-ec2-instance-for-s3-access/18546/3 "2015-06-19T06:52:53Z")

</div>

Thanks Benno,

I install Ansible and Boto in a virtualenv using pip, and then add the following to group\_vars/localhosts.yml, which is enough to ensure that the cloudformation, s3, and ec2 modules run on the localhost. Do I need to also install Boto on the remote?

`

# Do not use the system installed Python when running locally

ansible\_python\_interpreter: python

`

The exact set of packages is:

`

Jinja2==2.7.3  
MarkupSafe==0.23  
PyYAML==3.11  
ansible==1.9.1  
boto==2.38.0  
ecdsa==0.13  
paramiko==1.15.2  
pycrypto==2.6.1  
six==1.9.0  
wsgiref==0.1.2

`

regards

Andrew

---

<div class="post-metadata">

**Author:** ![Andrew\_Burrow](https://avatars.discourse-cdn.com/v4/letter/a/c5a1d2/32.png) [@Andrew\_Burrow](https://forum.ansible.com/u/Andrew_Burrow)\
**Post date:** [June 19, 2015, 7:23am UTC](https://forum.ansible.com/t/how-to-authorise-an-ec2-instance-for-s3-access/18546/4 "2015-06-19T07:23:03Z")

</div>

Just a follow up. I tried two more scenarios, the second being the boil-the-ocean approach 🙂

1. I deactivated the virtual environment, and reinstalled Ansible and Boto to /usr/local using Homebrew and Pip as follows:

`  
brew install ansible  
pip install boto==2.38.0

`

I then reran the playbook, and got the same error message, but again I was able to execute the s3 and cloudformation tasks locally.

1. I set my path to a minimum, uninstalled all the Homebrew python, and reinstalled Ansible and Boto using pip into the system as follows:

`  
PATH=“/usr/local/bin:/usr/bin:/bin”  
pip uninstall boto  
brew uninstall ansible  
brew uninstall python

curl -O [https://bootstrap.pypa.io/get-pip.py](https://bootstrap.pypa.io/get-pip.py)  
sudo python2.7 get-pip.py  
sudo pip install six  
sudo pip install boto  
sudo pip install ansible

`

I then reran the playbook, and got the same error message, but again I was able to execute the s3 and cloudformation tasks locally.

Thanks

Andrew

---

<div class="post-metadata">

**Author:** ![benno\_joy](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@benno\_joy](https://forum.ansible.com/u/benno_joy)\
**Post date:** [June 19, 2015, 7:30am UTC](https://forum.ansible.com/t/how-to-authorise-an-ec2-instance-for-s3-access/18546/5 "2015-06-19T07:30:20Z")

</div>

Hi Andrew,

Sorry if i am understanding this wrong, i assume you already have an ec2 instance which has an iam role attached which gives it access to download buckets/files from S3 right ? and in your playbook you have an s3 task which runs on this target server which has boto and python installed , so i am a bit confused as to why you would need to reinstall ansible boto etc… on your local macbook  
Probalby if you can attach your playbook it might make things clear.

- Benno

---

<div class="post-metadata">

**Author:** ![Andrew\_Burrow](https://avatars.discourse-cdn.com/v4/letter/a/c5a1d2/32.png) [@Andrew\_Burrow](https://forum.ansible.com/u/Andrew_Burrow)\
**Post date:** [June 19, 2015, 7:41am UTC](https://forum.ansible.com/t/how-to-authorise-an-ec2-instance-for-s3-access/18546/6 "2015-06-19T07:41:44Z")

</div>

No problems, I think you have the picture right, but might have missed my earlier question: do I need to install Boto on the target server?

So, yes:

- All playbooks are run on my laptop
- A playbook aws-start.yml first creates the EC2 instance. It operates on the localhost
- A playbook provision.yml then attempts to connect to the S3 bucket. It operates on the EC2 instance

Andrew

---

<div class="post-metadata">

**Author:** ![benno\_joy](https://avatars.discourse-cdn.com/v4/letter/b/73ab20/32.png) [@benno\_joy](https://forum.ansible.com/u/benno_joy)\
**Post date:** [June 19, 2015, 7:47am UTC](https://forum.ansible.com/t/how-to-authorise-an-ec2-instance-for-s3-access/18546/7 "2015-06-19T07:47:50Z")

</div>

Hi Andrew,

If the s3 task is running on the target server, then the provisioned instance needs to have boto installed. but if boto is not installed you should have got a message like “boto is not installed on this machine etc…” maybe an incomplete boto installation on target server ?

- Benno

---

<div class="post-metadata">

**Author:** ![Andrew\_Burrow](https://avatars.discourse-cdn.com/v4/letter/a/c5a1d2/32.png) [@Andrew\_Burrow](https://forum.ansible.com/u/Andrew_Burrow)\
**Post date:** [June 19, 2015, 7:56am UTC](https://forum.ansible.com/t/how-to-authorise-an-ec2-instance-for-s3-access/18546/8 "2015-06-19T07:56:58Z")

</div>

Ok. That is helpful. I will add that as an early task in the provision.yml playbook.

My confusion arises, because i have never installed Boto on the server instance, yet the task runs just fine with Ansible 1.8.4 if I also supply the AWS credential. It is a plain Ubuntu AMI, but perhaps it has the AWS tools installed in someway. I note that awscli depends on botocore.

thanks again

Andrew

---

<div class="post-metadata">

**Author:** ![Brian\_Coca](https://avatars.discourse-cdn.com/v4/letter/b/6de8d8/32.png) [@Brian\_Coca](https://forum.ansible.com/u/Brian_Coca)\
**Post date:** [June 19, 2015, 2:22pm UTC](https://forum.ansible.com/t/how-to-authorise-an-ec2-instance-for-s3-access/18546/9 "2015-06-19T14:22:36Z")

</div>

boto is needed on the machine executing the module, for most cloud  
operations this is normally your master/controller (local), but if you  
are trying to access s3 from a target/remote server, the module will  
run there, hence the requirement of boto on that machine.