The EC2 module’s docs show how to create several instances at once using the “count” variable, and register the results into another variable (“ec2”).
You can then iterate on the created instances and do stuff (like add them to an inventory group) using “with_items: ec2.instances”
However, I can’t figure out how to do the following :
Have the user provide a “names” list ( { “web1”, “web2”, “web3” } ). Or better yet, a naming prefix (“web”), a count (3) and a start index (1).
Create them using the ec2 module, register results into “ec2” to keep precious info like ec2.instances.public_dns_name.
Here’s the tricky bit : add a route53 DNS entry that CNAMEs together each entry in the “names” list with the public_dns_name values in “ec2.instances”.
I don’t believe with_nested will help with 3) and you can’t walk through the 2 lists together side by side either by having several with_items: statements, correct ?
Short of adding the feature to the ec2 module, how would you do that ?
Can you do something like :
include add_instance.yml
with_items: names
and then inside add_instance.yml use the ec2 module with count=1 and create the DNS entries from there ?
The EC2 module's docs show how to create several instances at once using the "count" variable, and register the results into another variable ("ec2").
You can then iterate on the created instances and do stuff (like add them to an inventory group) using "with_items: ec2.instances"
However, I can't figure out how to do the following :
1) Have the user provide a "names" list ( { "web1", "web2", "web3" } ). Or better yet, a naming prefix ("web"), a count (3) and a start index (1).
2) Create them using the ec2 module, register results into "ec2" to keep precious info like ec2.instances.public_dns_name.
3) Here's the tricky bit : add a route53 DNS entry that CNAMEs together each entry in the "names" list with the public_dns_name values in "ec2.instances".
Can you give an example for what you are trying to convey here? I'm a bit confused.
I’m trying to create a fleet of (for instance) web servers on EC2, and would like to give them meaningful ansible hostnames (web1, web2, etc) instead of the AWS generated public DNS names (ec2-xxxxx.compute-1.amazonaws.com)
I’d also like to create a CNAME DNS record (using the route53 module) pointing their chosen name (web1, web2, etc) to the EC2 DNS record (ec2-xxxxx.compute-1.amazonaws.com). The latter is accessible in the « instances.public_dns_name »
Ideally, I would just need to provide the ec2 module with a list of hostnames for the instances it’s going to create (internally it just needs to set the « Name » EC2 tag), and I’d get back the « item.hostname » (web1,web2) alongside each « item.public_dns_name » (ec2-xxxxx.compute-1.amazonaws.com) when using “with_items: ec2”
The more I think of it, the more it sounds like just a few changes are needed in the ec2 module. I can try and have a go at it, but I’d like to know if there’s something similar already.
I know (and mentioned) I’ll need to use the route53 module, that’s not the difficulty here.
I did some more research and it looks like ‘with_together’ is what I was looking for.
If anyone is looking to do the same (it’s a quite common provisioning pattern) :
provide an « instances_names » list
create the required number of instances using the ec2 module, register the result.
Use ec2_tag, add_host and route53 for, respectively: setting the AWS Name tag, adding the host to the inventory with its desired hostname (not the default EC2 name), adding a CNAME DNS entry for hostname → EC2 public_dns_name
-For each of the modules above, use « with_together » to iterate through both the created instances list (returned by the EC2 module), and the hostnames list you provided.
This would be slightly easier if the ec2 module would take an optional « instances_names » list as a parameter; and used it to set different EC2 Name tags for each of the created instances.
I can look at adding this functionality if it’s likely to be accepted and merged in. What do you think ?
Very interesting talk and this is probably the better approach, thanks.
You do need to address individual servers from time to time though : ssh’ing into them to debug something, for instance.
In this case, finding and copying the public_dns_name for a box sounds like a pain, and a big usability regression from a human-readable naming convention and DNS CNAMEs.
Am I missing a clever way to use ansible to provide « vagrant ssh » functionality basically ?
Or do you know of any wrapper tools to do something similar ?
It uses Ansible’s EC2 inventory plugin to do the lookup of public DNS name (or whatever you configure it to).
One further tip. In a distributed environment, debugging can be tough because you don’t know which server is causing issues. Get all of the logging out of the instances and into some central and searchable location. There are plenty of SaaS providers (Loggly, Splunk, Sumologic, Papertrail) and open source (logstash, etc) options available.
But your ssh example shows how a unique name is still unavoidable sometimes, be it Tags or DNS based.
If I left instances with their EC2 birth names, I couldn’t just make up « ansible-ec2 ssh --name web1 » when I need to log into a web server, I’d have to « ansible-ec2 list » first with a raw group name to find web servers, then pick one and ssh into it.
Also, some tools just rely on a nice hostname being set (the New Relic server dashboard would be unreadable with ec2-xxxx host names all mixed up)
RabbitMQ for instance, names its task queues after the current hostname and warns against changing it. I know it’s bad practice and not autoscaling friendly, but it’s just how it is in some occasions.
So, I’ll forget about CNAMEs in Route53 but I think my original feature request is still relevant : being able to pass a list of names to the EC2 module to be added to the Name tag of each created instance.
If I submitted such a patch, would you guys merge it ?
Because that changes the game completely making the registered ec2 instances info outdated (at least the IP-related one).
How about cycling through Route53 with those EIPs? Has anyone done that successfully? Don’t want to hijack this thread, but I find this rather relevant.
I know some would indicate the ec2_facts module, registering the output from ec2_eip or using the add_host module but I couldn’t do it in any way so even a pseudo-code would be appreciated.
I totally get what you are looking to achieve. Perhaps you’ve achieved it already in the past year. Assuming that you are happy with specifying a start index = x and a count = N to produce machines with names containing x, x+1, x+2, … x+N, I think the following playbook example will be instructive. I had the same requirement and this is what I was able to produce:
I must also add that I’ve got a central group_vars/all file from which I pull all the variables like region, instance types, environment specific load balancer names, etc.