Structuring and managing multiple unrelated projects

Archives Ansible Project
1

Hey

I’m really struggling to find a solution or “best practise” that enables me to use Ansible to manage multiple different projects, while still being able to reuse roles between them to target hosts from all projects in a play.

Every tutorial or document I read seems to assume you are provisioning one project with a handful of web and database servers in various regions or whatever. What we have currently is a single server per client/project and the way I have been using Ansible so far is:

`
playbooks/

  • files/
  • public_keys/ ← common keys used on multiple projects/servers
  • galaxy_roles/
  • group_vars/
  • all/
  • droplets/
  • host_vars/
  • project1/
  • vars.yml
  • vault.yml
  • project2/
  • vars.yml
  • vault.yml
  • roles/ ← custom roles, as reusable and broken down as I can make them
  • templates/
  • iptables/
  • project1/rules.j2 ← each project may require different iptable rules
  • project2/rules.j2
  • project1.yml
  • project2.yml
    `

It doesn’t scale well. I know that if “project1” was suddenly comprised of more web servers for example, then I should probably be looking at group_vars instead of host_vars. I haven’t experimented with groups of groups. I also have to check three places to find the configuration used for a server (all, droplets, host_vars/project (and I guess the role defaults too really)).

I thought about having a separate folder per project, each with its own ansible.cfg and inventory to allow for future scaling, modifying the roles_path to find the common/reusable roles and galaxy roles. This feels neat, but then I can’t target all servers in multiple projects (for example to apply an ad-hoc patch). I think the public_keys and iptables setup would also need to be altered so that these files are within the roles somehow.

My setup is starting to feel unwieldy and cumbersome and I’d like to get to the point where I have a solid foundation where Ansible is helping me more than being a hindrance.

Any thoughts or suggestions or links I may have missed much appreciated.

Thanks
Jamie

2

Hi,

I don’t think I have an answer for you, but faced with a similar problem (being able to reuse roles and yet have separate projects) we started investigating the following solution. At this stage we’re only trying this with one and a bit of (smallish) projects so I’m unsure how well this is going to work yet

  1. separate ansible.cfg for each project with a list of 2 role paths (project and shared)
  2. a repo of ‘shared roles’ (with no playbooks)
    shared roles look for its data in specific locations:
  • its own default/vars directories
  • predefined project directories (generally host_vars/role_name/…) this is manually loaded
    shared roles are a submodule in git
  1. actual repo for the project with the usual setup

Two things we’ve noticed so far:

  • shared roles have to be truely reusable, abstracted out and idempotent - takes more effort
  • the behaviour of shared roles must be well defined: all input parameters must be validated and when they fail they must leave good trail of what actually when wrong

I’m also keen to hear how others scale up their setups. One of the issues we seem to have is the multitude of top-level playbooks - we try to keep them separate by using naming conventions (like object-action-subject) but that only helps to a degree.

kind regards
Pshem

3

I just tried the alternative approach that I was thinking about and this is working however two problems

  • A lot of configuration repetition between projects that are different but similar. I guess it’s a balance between finding configuration in multiple places and less repetition vs. one config file with repetition.
  • No way to target all of the hosts across all our projects to apply a patch or similar ad-hoc

`

  • playbooks/
  • (as before)
  • project/
  • group_vars/
  • all/
  • web/
  • templates/
  • iptables/
  • web/
  • ansible.cfg
  • hosts
  • site.yml ← just includes web.yml
  • web.yml

`

4

So after converting my structure to the "each project has its own ansible.cfg and inventory" the worst issue I don't appear to be able to solve is targeting hosts in different projects. If I need to run an ad hoc command on all hosts, I literally have to enter each project directory and run the command. I could script around it somehow or maybe there's some clever shell expansion I could use, but it feels like going against what Ansible is good at: running against multiple hosts automatically.

As most of my projects only have a single host each, it's not very useful at this stage. Sure, if I need to scale the project I may need a few new hosts but right now it feels like each project should be a group in one big project.

5

I could duplicate the hosts in each project in to one central hosts file (yuk) and at least that way I could run an ad hoc command and target them. I couldn't rely on group_vars or host_vars though - I'd have to pass in parameters in a playbook directly or on the command line I guess.

6

Hi,

I think it’s down to your work split/structure. In our case each project operates on separate set of hosts. So we have separate inventories for each of them. Perhaps there’s some sort of logical grouping structure you could create for your hosts and run playbooks based on those groups (with a single big shared inventory).

kind regards
Pshem

7

They are at the moment at least webservers each hosting a project or group of projects relating to a particular client. It makes resource allocation and billing easier. It affords us the facility to throw hardware resources or scale up a client's needs. But although each server is essentially a LAMP stack, some use different software or have different requirements and would need a playbook of their own.

Managing all projects, their webservers, future additional servers and DB servers etc, plus prod/staging/testing versions in one inventory isn't ideal either.

8

You could just keep your individual inventories for each client but then use a directory containing symlinks to all the client inventory files.

I use this trick as I have 2 environments that run inside the same physical datacenter. Mostly I want to make changes to just one environment at a time, but for stuff like patching it makes sense to hit all the hosts in the datacenter.

Only slight downside is if you use group_vars then you have to symlink to the group_vars as well.

Jon

9

Yeah, I think I’d need to create symlinks to all the group_vars too. Actually, I’m not too sure how it could work because each inventory file looks like this:

[web] somehost

[web] some_other_host

10

I think the hosts would get merged into a single group called web, so that in itself might not be a problem.
Where you might fall down is if you have different group var settings for hosts belonging to the same group names.

If that’s the case you might be able to move things around a little or perhaps have a different group_vars file in the dir that just contains common settings across your hosts.

11

So years later I’m still struggling with this :slight_smile:

The way I’m using Ansible at the moment currently gives me the ability to easily target any host or group of hosts that make up any web project we have. It works and it feels like I’m utilising one of the major selling points of Ansible in this regard. The issues arise where projects diverge from the “standard setup” and require additional roles or setups applied to them and for this use case, my setup doesn’t feel right and I’m unhappy with it.

I use Ansible to bringing up a basic type of machine and let other processes/configuration tools/developers deploy their project to it. Perhaps I need to rethink what I’m using Ansible for i.e. is it to bring up a basic environment or is it to fully provision an environment and project and its dependencies.

Current structure:

files/ (public ssh keys and the like) galaxy_roles/ group_vars/ (each project has its own group vars) all project1_web/ project1_db/ project2_web/ project2_db/ project3 backups host_vars/ (not used) roles/ templates/ (just used for firewall rule templates) utils/ (adhoc commands that I sometimes use) ansible.cfg hosts play_webservers_lamp.yml play_webservers_lemp.yml play_project3.yml play_backups.yml requirements.yml

Hosts:

`
; Webservers

[webservers:children]
webservers_lamp
webservers_lemp

[webservers_lemp:children]
project1_web
project2_web

; Backup systems

[backup_systems]
backups

; Individual hosts

[project1_web]
project1_web1 ansible_host=1.1.1.1

[project2_web]
project2_web1 ansible_host=2.2.2.2

[backups]
backups1 ansible_host=9.9.9.9

`

In this case the idea is, project1_web is a group of identical web servers belonging to project1. Similarly, I imagine I’d create project1_db for a group of database servers belonging to the same project. project1_web and project2_web in this case might be LEMP servers which are only differ in subtle ways, which can be handled using group configurations - think resource allocation e.g. swapfile size, memcache limit, php max memory etc. but also which php packages to install for a project and firewall rules.

Most of the projects either fall under the webservers_lamp or webservers_lemp groups, which have slightly different plays.

Occasionally, we’ll get an odd-ball that doesn’t fit in to these two groups neatly e.g. project3.yml and backups.yml which may be completely different setups or not web servers at all.

So the goal is to achieve a good separation for each project and environment, but also to leverage Ansible’s main selling point as I see it, which is that if I need to adjust a global configuration parameter and apply that to all systems in every project, that’s still possible. Likewise, if I wanted to run an adhoc command across every system I can, instead of potentially needing to manually run a play from every project’s directory. If I need to do that, I may as well not use Ansible.

If anyone has any better ideas of how I could improve my setup I’d love to hear them.

Thanks,
Jamie

12

This might be accomplished by ansible-pull periodically triggered from cron.

Cheers,

  -vlado