(a) can you please show the line from your playbook where you are using the user data variable?
(b) with the above, how are you determining the way it fails? I.e. what does failure look like?
I got the following error:
`
ERROR: Syntax Error while loading YAML script, /etc/ansible/playbooks/search/roles/aws/tasks/main.yml
Note: The error may actually appear before this position: line 13, column 17
state: running
user_data: “”"#!/bin/bash
^
This one looks easy to fix. It seems that there is a value started
with a quote, and the YAML parser is expecting to see the line ended
with the same kind of quote. For instance:
when: “ok” in result.stdout
Could be written as:
when: ‘“ok” in result.stdout’
or equivalently:
when: “‘ok’ in result.stdout”
`
I don’t believe YAML takes Python-like docstrings.
So you would need to start with a single or double quote and escape any internal quotes as need be.
The “file” lookup plugin may also be helpful.
The file lookup plugin is your friend.
I’m using the following task for launching Windows instances and configuring PowerShell remoting:
- ec2:
aws_access_key: ‘{{ aws_access_key }}’
aws_secret_key: ‘{{ aws_secret_key }}’
region: ‘{{ win_ec2_region }}’
instance_type: ‘{{ win_ec2_instance_type }}’
instance_tags:
Name: ‘{{win_ec2_name_prefix}}-{{item.name}}’
group: ‘{{ win_ec2_security_group }}’
key_name: ‘{{ win_ec2_key_name }}’
image: ‘{{ item.id }}’
user_data: “{{ lookup(‘file’, ‘win_ec2_user_data’) }}”
exact_count: 1
count_tag:
Name: ‘{{win_ec2_name_prefix}}-{{item.name}}’
wait: yes
with_items: win_ec2_images
register: win_ec2
My win_ec2_user_data file contains:
Hi Chris,
What’s your approach for logging in to your Win boxes using WinRM after this step?
Ansible docs direct you to configure a vars file with ansible_ssh_pass set however every ec2 instance is going to have a different password. I’ve created a module for using boto to get the ec2 password but i dont know how to chain this through to then use it for a following play which would configure my windows instances.
I’m planning to get some of these roles onto Galaxy, but until then…
The next tasks in that particular play are:
- name: wait for instances to listen on port 5986 (winrm https)
wait_for:
state=started
host={{ item.tagged_instances[0].public_ip }}
port=5986
with_items: win_ec2.results
when: win_ec2_images and win_ec2 is defined
- name: obtain windows passwords for instances
ec2_win_pass:
aws_access_key: ‘{{ aws_access_key }}’
aws_secret_key: ‘{{ aws_secret_key }}’
region: ‘{{ win_ec2_region }}’
instance_id: “{{ item.tagged_instances[0].id }}”
private_key: “{{ lookup(‘file’, win_ec2_private_key) }}”
with_items: win_ec2.results
register: win_ec2_passwords
when: win_ec2_images and win_ec2 is defined
- name: add_host
add_host:
name: ‘{{ item.item.tagged_instances[0].public_dns_name }}’
groups: ‘cloud,ec2,windows,{{win_ec2_name_prefix}}-{{ item.item.item.name }}’
ansible_ssh_host: ‘{{ item.item.tagged_instances[0].public_ip }}’
ansible_ssh_port: 5986
ansible_ssh_user: ‘{{ item.item.item.user }}’
ansible_ssh_pass: ‘{{ item.password }}’
ansible_connection: ‘winrm’
with_items: win_ec2_passwords.results
when: win_ec2_images and win_ec2_passwords is defined
The second task is using my module from https://github.com/ansible/ansible-modules-core/pull/378
You can also save these new hosts to an inventory file (using template module).
After the play above runs, I can then run another play to ping the newly-launched Windows hosts:
- name: ping new windows instances
hosts: windows
gather_facts: false
tasks:
- action: win_ping
Hope that helps!
It is – if one wishes to supply text-based user data. Unfortunately, I have a requirement to provide a binary blob of user data, and the file lookup plugin does not seem to handle that – looking at the source code, it expects the file to be a UTF8-encoded text file.
What I’d like to do is make use of FreeBSD’s configinit capability, which can treat the user data as a tar file, extract it, and then process each of the contained files. This is documented here:
http://www.daemonology.net/blog/2013-12-09-FreeBSD-EC2-configinit.html
As I mention in the comments on that page, I did try passing the contents of a base64-encoded tar file (via the file lookup plugin) in as the user_data parameter, but (unsurprisingly), that gets passed-through as-is (i.e. still base64-encoded) to the ec2 instance user data, whereas I need the user data to be set to the raw binary data of the unencoded tar file.
Now, the Ansible ec2 module documentation specifically says that the user_data parameter is an ‘opaque blob of data which is made available to the ec2 instance’, so it is obviously anticipating that directly passing a binary blob should be possible. The question, though, is how?
Any help would be greatly appreciated. I am new to both Ansible and EC2, so it is entirely possible that I am missing something blindingly obvious.
Thanks in advance!
Incidentally, I also tried using the file lookup plugin on a base64-encoded tar file, and then filtering that with Jinja2’s b64decode filter. That also fails, though I think it does so at a later stage than trying to use the file lookup plugin on a binary file. Using the b64decode filter on a looked-up base64-encoded file, one gets an error whose stack trace ends something like this:
File “/usr/local/Cellar/ansible/1.8.2/libexec/lib/python2.7/site-packages/ansible/utils/template.py”, line 362, in template_from_string
res = jinja2.utils.concat(rf)
File “”, line 7, in root
UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xfd in position 0: ordinal not in range(128)
I then thought I’d do a little experiment, and provide the user data as a literal (copy-and-pasted) string of base64-encoded data, again filtered with the Jinga2 b64decode filter, like this:
user_data: “{{ ‘/Td6WFoAAATm1rRGA … VZiascRn+wIAAAAABFla’ | b64decode }}”
That took the file lookup plugin out of the equation, but resulted in the same error.
I next tried the pipe lookup plugin (with ‘cat my-binary-file’), but that also fails in the same way as the file lookup plugin. Looking at the source code, the pipe lookup plugin follows the file lookup one and unconditionally tries to decode the output from the command as a UTF-8 encoded string.
I provisionally conclude from all this that Ansible generally doesn’t want to handle binary parameter values. That seems limiting. I suppose one solution to my specific problem (without fixing the wider issue) would be for the ec2 module to accept something like a user_data_file parameter (as an alternative to user_data), which would specify a local file whose content was to be used without any processing at all as an opaque user data binary object. In fact, that’s pretty much exactly what I want, as I could give it the FreeBSD configinit .tar file directly.
I am still hoping that someone has a solution that will enable binary user data to be provided to an instance.
Again, my thanks in advance for any assistance.
One issue may be that boto is encoding unicode user data into utf-8, then base64-encoding it:
https://github.com/boto/boto/blob/develop/boto/ec2/connection.py#L927
So, if we can somehow pass a str() instance to boto instead of a unicode() instance, it’ll skip the utf-8 encoding step.
I’ve implemented a version of the ec2 module that accepts a user_data_b64 parameter as a base64-encoded string of the user data, and decodes it into a str() instance before passing to boto:
You can read the local binary file into a base64-encoded string using the slurp module, then pass that blob to the ec2 module. The slurp module needs an absolute path, so you can use role_path to specify a path relative to a role, or playbook_dir relative to your top-level playbook.
Yaml has a notation for multiline string. I set user data like so:
user_data: | #!/bin/bash echo "Defaults:{{admin_user}} !requiretty" > /etc/sudoers.d/disable_requiretty
Note the pipeline character.
`
# Join multiple lines without new line
value: >
part 1
part 2
# Join with newline
value2: |
line 1
line 2
`
Chris, thank you very much for looking at that. I have tried your modified ec2.py but, unfortunately, even using it exactly as you describe, the user data is still being stored on AWS in its base64 form. From within the instance:
fetch http://169.254.169.254/latest/user-data
cat user-data
/Td6WFoAAATm1rRGAgAhARYAAAB … /sCAAAAAARZWg==
base64 -d user-data test.tar.xz
tar tvf test.tar.xz
-rw-r–r-- 0 djn staff 2 Dec 26 10:40 1.txt
-rw-r–r-- 0 djn staff 2 Dec 26 10:40 2.txt
-rw-r–r-- 0 djn staff 322 Dec 24 12:03 configinit
By the way, looking at line 930 of the boto connection.py source file you referenced, I don’t understand why boto is base64-encoding a parameter that its own documentation (on lines 774-776) indicates is apparently already to be base64-encoded. (Note, however, that I am not familiar with the boto source.) Still, since you are passing in a decoded version of the Ansible user_data_b64 parameter, I’d have thought what you’ve done ought to work. Empirically, though, it doesn’t seem to.
Finally, perhaps I should note that to test this, I dropped your ec2.py over the top of the existing ec2.py in a stock Ansible 1.8.2 installation that was installed by Homebrew on my OS X machine. This installation of Ansible is using boto 2.34.0, I believe.
Thank you again for your help!
Chris, please ignore my previous response, which was premature. I had misinterpreted the (perhaps slightly ambiguous) documentation for the slurp module, and thought that it needed to be given a base64-encoded file. Apparently, it does not. If I give it a raw .tar.xz, your ec2.py revision seems to work perfectly, thank you!
Perhaps an example in the documentation for the ec2 module using slurp and user_data would be in order, however, before you submit a pull request for your modifications to be incorporated into Ansible proper?
Many thanks again for your solution, let’s hope it can be incorporated into the next Ansible release.
I’m having trouble using this notation.
- ec2_lc:
name: “{{env}}-render-premium-{{gitsha}}”
…
user_data: |
{
“services”: [“worker-render-premium@4100”],
“autoScalingGroupName”: “{{env}}-render-premium”
}
but when I do a curl http://169.254.169.254/latest/user-data to read it back I get:
{‘services’: [‘worker-render-premium@4100’], ‘autoScalingGroupName’: ‘staging-render-premium’}
The loss of whitespace I can deal with, but translating double quotes into single quotes makes this invalid JSON and breaks things.
I’m using ansible 1.9.4. Even with -vvvv user_data isn’t printed.
Hey,
You want to put your data in a variable, let’s say in /defaults/main.yml and then invoke that from the ec2_lc module:
- ec2_lc:
name: “{{env}}-render-premium-{{gitsha}}”
…
user_data: “{ {my_user_data }}”
I did that exact thing for Launch Configs and it works prefectly.
Other than that you may try to rewrite your do to include the pipe WITHIN the brackets. But I wouldn’t recommend that as a best practice.
- ec2_lc:
name: “{{env}}-render-premium-{{gitsha}}”
…
user_data:
{ |
“services”: [“worker-render-premium@4100”],
“autoScalingGroupName”: “{{env}}-render-premium”
}
my_user_data: |
#!/bin/bash
echo “Defaults:{{admin_user}} !requiretty” > /etc/sudoers.d/disable_requiretty
And then use that variable from within the module:
Hello Guys, new tot this Group, trying to provision an EC2 Windows using Ansible with user_data.
And your question is…?