Best scalable approach to bootstrapping an EC2 server from within AWS Cloudformation?

I am relatively new to Ansible (and config. management in general) so please excuse me if I sound confused. I encourage you to correct me or clarify me if I am wrong.

I have setup an AWS Cloudformation template through AWS and not Ansible, have defined my infrastructure, auto-scaling group and launch configurations for out-scaling servers in a VPC. I understand I have a few choices to bootstrap the servers as they are launched: (I’m not 100% sure if these are the correct ways to implement the choices)

  1. Use metadata & cloud-init in Cloudformation to run bash scripts directly

  2. Pre-bake AMI’s with packages and run playbooks locally when the server is launched.

  3. Install packages, Ansible, copy playbooks from a private repo, and run playbooks locally

  4. Install packages, Ansible, and use ‘ansible-pull’

What I am confused about is that choices #2,3,4 require Ansible to be installed just to run a local playbook. But would the servers even use Ansible after it’s initial bootstrap? Is it really necessary to have Ansible on the remote server (if so, could you explain a use case)? I’m assuming after it’s ready I would no longer use Ansible on the remote server and instead provision/update the servers from my control server.

I want to make it automated so that I don’t have to:

  1. install Ansible on the remote server if I don’t have to or it can be done another, more scalable, way.

  2. SSH into the remote server to install all the initial packages and code manually.
    I was considering choice #1 because I can install packages, start nginx, and my application through bash directly in the cloud-init/metadata. However I am assuming it’s not very dynamic.

What are your thoughts/opinions/advice? Thank you for your help in advance!

I’ve been implementing a bit of 1, 3, 4 to bring up ASG instances.

I, like many people dislike having unnecessary software on my hosts (instances) so thought this could be another way to carry out the same process - Running Ansible from AWS Lambda
https://medium.com/@jacoelho/ansible-in-aws-lambda-980bb8b5791b#.u8t6gsbvp

Though I haven’t yet figured out how to do this yet with the 2.0 api of Ansible. It would be really nice because then you could further restrict the access/rights that the instances have. It’s really annoying when you aren’t using an elb and have to then also update route53 values when the instances comes up.

How are you implementing these? I was thinking of taking advantage of AWS’s Lifecycle Hooks to trigger a Lambda function to have Ansible run playbooks on the pending server but I think that may be a little too much.

  1. Use metadata & cloud-init in Cloudformation to run bash scripts directly

  2. Install packages, Ansible, copy playbooks from a private repo, and run playbooks locally

  3. Install packages, Ansible, and use ‘ansible-pull’

I get the lc user data script to install ansible, pull from an s3 bucket and then just run Ansible locally (not ansible-pull). I have it run a bootstrap playbook which get’s the instance tags from the metadata. In the ASG configuration I have a tag, ansible_host_group which includes the host groups this instance will belong to in the Ansible (dynamic ec2) inventory. The bootstrap playbook uses add_host to add the localhost i.e. the instance to the ansible_host_group group, site.yml is then included. site.yml includes the other application tier playbooks, webserver.yml, database.yml etc. The hosts: value restricts what instances run the plays/roles.

webserver.yml

Cool, so I feel like I’m on the right track! I was thinking of doing it similarly. Thank you for replying so quickly.