EC2 Autoscaling

Hi,

I’m studying the possibility of using ansible instead of puppet.
I’ve been testing it lately and, so far, I think I’am falling in love with it…
It fulfills almost all the needs I have, but yet there is one missing thing. I am not sure if it means ansible cannot do that or that I don’t know how, so I ask.

I would like to do an autoscaling in EC2. I mean a complete autoscaling so if a new php backend starts automatically I would like ansible to get it ready.
I read about the “pull mode”, and as I understand it it is half way what I need as I can pull the playbooks from git, but I didn’t found yet a way to identify which kind of machine must be created (database, frontend, backend…).
I suppose one approach could be to have different AMIs with different configs pulling from different git sources,… but it is not elegant at all and I would find myself maintaining a lot of AMIs.
Another option I though about is to have the start script (the one installing ansible and pulling from git) to check which kind of group the EC2 instance is in but I didn’t try it yes so I don’t really know what it implies.
So, I decided to ask if someone already did this, or if someone knows some best practices in that scenario…

Thanks!

There is an ec2 inventory script that can be used instead of ansible-pull, and you can assign tags to the instance when they’re created to target them with more specific arguments.

Beyond that, the best thing to do would be creating custom AMI’s for your instances using aminator or some other tool. The custom AMI’s could just contain a local facts file (http://www.ansibleworks.com/docs/playbooks2.html#local-facts-facts-d, available in 1.3) containing facts about how the instance should be configured.

Hi Dan!

Yep, autoscaling from EC2 can be done in a couple of ways. At a basic level, you can use “ansible-pull” set up to run at first boot, to pull the latest content from git.

A better way though is to use AWX provisioning callbacks, where the guest can be configured to make a simple curl request, where when prompted the management server will reach out and configure the guest that requested configuration.

The way this works is it will run the given requested playbook (for example site.yml) and that way all the group information can be totally dynamic based on what groups you put the machines in, which that too can also come from your EC2 inventory and how you have tagged the instances.

–Michael

Hi James,

See what I just posted below about how to do this without custom AMIs with AWX :slight_smile:

–Michael

Also this might help you as well.

http://www.holovaty.com/writing/aws-notes/

I am embarking on this for both web and an xml database autoscaled. It will be all ansible driven.

Lots of ways to do this.

Eric Palmer

Send me a email if you want to compare notes. eric [at] ericfpalmer [.] com

If you want to use the same AMI as the base for multiple machine types using ansible-pull, then one thing you can do to specify the machine type is in the LaunchConfig of the AutoScaling Group, add a section to the User Data and that runs ansible-pull with the machine type (database, front end, etc) playbook.

Here is the doc on User Data:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AESDG-chapter-instancedata.html#instancedata-user-data-retrieval

Depending on your OS, you can even embed scripts into the User Data which are run on boot. This isn’t just for Ubuntu, Amazon Linux supports this as well. Check your OS:
https://help.ubuntu.com/community/CloudInit

With those 2 things, ASG with Ansible should be working well.

We’ve been experimenting with using Ansible to automate ASGs. So far, we have a series of scripts to automate the creation of AMIs (a “bakery” to use the Netflix terminology).

I plan to write a blog post once we have a few more ansible management scripts up and running.

All the best,

~ Christopher

Thank you all guys!

I’ll have to give it a try!
I saw the User Data way but rite now I’m using Scientific Linux and I think it is not supported. Maybe I’ll have to think about changing to Centos,…

Maybe it would also be a way if before executing ansible pull mode we would export the result (from the instance) of :
curl http://169.254.169.254/latest/meta-data/security-groups
And maybe use it form within ansible itself.
Or even make a different call to git depending on this variable.
Someone tryied it?

More to explore…

Hi Dan,

You can still use User Data even if your OS doesn’t support CloudInit. Just pass in the data you want to share between all instance in the ASG, and you can access it like this:

curl http://169.254.169.254/latest/user-data

This is more flexible than checking security groups.

Thanks Peter,

Yes, it is clearly much more flexible! I’ll give it a try.

Did anyone come up with a cloudinit script that would invoke ansible-pull with a given role? Having trouble figuring this part out by myself.

You can pass any script you like to cloud-init as part of user-data, and it will be executed. Just start the script with #! and it will be executed as the root user.

Here is a short screencast explaining how to do it by passing in a variable from CloudFormation:

http://answersforaws.com/episodes/4-user-data-cloud-init-cloudformation/

It doesn’t touch on ansible-pull, but it is trivial to do once you have the process working.