# multi-role organization and re-use **URL:** \ **Category:** Ansible Project\ **Created:** [June 19, 2013, 9:16pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990 "2013-06-19T21:16:14Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![James\_Martin](https://avatars.discourse-cdn.com/v4/letter/j/f14d63/32.png) [@James\_Martin](https://forum.ansible.com/u/James_Martin)\ **Post date:** [June 19, 2013, 9:16pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/1 "2013-06-19T21:16:14Z")
The purpose of this message is to come up with a best practices for role/playbook/library layout that encourages re-use and logically separates projects & roles into their own repositories. Just as an aside, I'm not a git expert by any means, so I'm always looking to simplify my workflow and organization. If I'm overcomplicating things, by all means let me know! \*TL;DR - what's the best way to re-use roles and playbooks across projects? Let's say I have project voltron. Project voltron uses the following roles -- web and db Of course the web and db roles all live in source control with the general layout: ansible-web-role: . ├── files ├── handlers ├── library ├── tasks ├── templates └── vars These roles are also used outside of the voltron project, so there's also a repo that uses ansible-web-role as a subrepo and would look something like this: ansible-web: . ├── bender.yml ├── roles │ └── web │ ├── files │ ├── handlers │ ├── library │ ├── tasks │ ├── templates │ └── vars └── setup\_web.yml As you can see there are some additional playbooks that I am using, and the roles/web subdirectory is just a git subrepo, which means any changes that happen to the ansible-web-role project can be pulled down into the ansible-web project. Also changes made in the roles/web directory can get pushed up to the ansible-web-role repo. Now to tie it all together in the voltron project: . └── roles     ├── db     │ ├── files     │ ├── handlers     │ ├── library     │ ├── tasks     │ ├── templates     │ └── vars     └── web         ├── files         ├── handlers         ├── library         ├── tasks         ├── templates         └── vars Here the web and db roles are subrepos of the voltron-repo. The problem that comes into play is what if I want to re-use the bender.yml and setup\_web.yml playbooks from the ansible-web repo? One (hacky) solution I can think of is running a script that would copy those playbooks into the root of the voltron project, or checkout the ansible-web repo Another idea is something like this. Place all the subprojects (roles and associative playbooks) into a "subprojects" folder that lives at the root of the project. Subrepo all of dependent projects into the "subprojects" folder. Symlink the roles and playbooks to their proper locations. It would end up looking something like this: voltron\_project: . ├── bender.yml -\> subprojects/web/bender.yml ├── roles │ ├── db -\> ../subprojects/db/roles/db │ └── web -\> ../subprojects/web/roles/web ├── setup\_db.yml -\> subprojects/db/setup\_db.yml ├── setup\_web.yml -\> subprojects/web/setup\_web.yml └── subprojects     ├── db     │ ├── roles     │ │ └── db     │ │ ├── files     │ │ ├── handlers     │ │ ├── library     │ │ ├── tasks     │ │ ├── templates     │ │ └── vars     │ └── setup\_db.yml     └── web         ├── bender.yml         ├── roles         │ └── web         │ ├── files         │ ├── handlers         │ ├── library         │ ├── tasks         │ ├── templates         │ └── vars         └── setup\_web.yml Symlinks in git make me feel icky, but this is just to get a conversation started. This keeps the playbook outside of the role (as it's not a roles purpose to house playbooks), but is messier. Another thought would be to just include a subdir in the role that provided playbooks that used that role exclusively. Then just those playbooks could be symlinked into the project. Something like this: . ├── bender.yml -\> roles/web/playbooks/bender.yml ├── roles │ ├── db │ │ ├── files │ │ ├── handlers │ │ ├── library │ │ ├── playbooks │ │ │ └── setup\_db.yml │ │ ├── tasks │ │ ├── templates │ │ └── vars │ └── web │ ├── files │ ├── handlers │ ├── library │ ├── playbooks │ │ ├── bender.yml │ │ └── setup\_web.yml │ ├── tasks │ ├── templates │ └── vars ├── setup\_db.yml -\> roles/db/playbooks/setup\_db.yml ├── setup\_web.yml -\> roles/web/playbooks/setup\_web.yml └── site.yml If you made it this far, thanks! James ---
**Author:** ![GW1](https://avatars.discourse-cdn.com/v4/letter/g/c89c15/32.png) [@GW1](https://forum.ansible.com/u/GW1)\ **Post date:** [June 19, 2013, 10:38pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/2 "2013-06-19T22:38:26Z")
Hi, In my opinion you are over-complicating in trying to have re-usable playbooks. If you only have re-usable roles, than it is almost obvious how the structure should look like. Greetings, gw ---
**Author:** ![James\_Martin2](https://avatars.discourse-cdn.com/v4/letter/j/b9e5f3/32.png) [@James\_Martin2](https://forum.ansible.com/u/James_Martin2)\ **Post date:** [June 19, 2013, 10:53pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/3 "2013-06-19T22:53:20Z")
Gw, Yes, the structure if I'm only re-using roles is obvious. It's just that some of the playbooks I have are complex and I'd like to only have only one place where they need to be updated. - James ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [June 19, 2013, 11:14pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/4 "2013-06-19T23:14:05Z")
Right, roles are the main unit of reusability. If you want to include playbooks inside of other playbooks, you of course can. Ultimately, I hate to be the “TLDR” guy, but it would be useful if you could boil down the question to the most salient point fo me. ---
**Author:** ![Chris\_Bennett](https://avatars.discourse-cdn.com/v4/letter/c/e19adc/32.png) [@Chris\_Bennett](https://forum.ansible.com/u/Chris_Bennett)\ **Post date:** [June 20, 2013, 12:47am UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/5 "2013-06-20T00:47:55Z")
Hi list, I think I'm dealing with a similar problem where I'm managing multiple independent sites with servers, that may resuse roles, but the primary 'site.yml' as well as a subdir with site specific vars is located in a site specific subdir, and I use a separate common location for reusable roles & playbooks. e.g instead of as what's documented at [http://www.ansibleworks.com/docs/playbooks.html](http://www.ansibleworks.com/docs/playbooks.html)   site.yml   [..]   roles/     [..] I am trying to manage it like this:   common/webserver.yml   common/roles/   common/roles/[..]   site1/site.yml   site1/vars/{{variable}}.yml   [..]   site2/site.yml   site2/vars/{{variable}}.yml   [..] In siteX/site.yml, I would like to specify: ---
**Author:** ![James\_Martin2](https://avatars.discourse-cdn.com/v4/letter/j/b9e5f3/32.png) [@James\_Martin2](https://forum.ansible.com/u/James_Martin2)\ **Post date:** [June 20, 2013, 2:06am UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/6 "2013-06-20T02:06:07Z")
> Right, roles are the main unit of reusability. > > If you want to include playbooks inside of other playbooks, you of course can. > > Ultimately, I hate to be the “TLDR” guy, but it would be useful if you could boil down the question to the most salient point fo me. There was a TLDR, granted it was after the first paragraph so maybe it was too late by then 🙂 James ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [June 20, 2013, 2:49am UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/7 "2013-06-20T02:49:45Z")
I tend to understand things better when we talk in terms of use cases rather than syntax, I guess. If you need different variables for different sites, this is exactly what “group\_vars/” is for, so this may be a question of trying to apply the wrong solution (“roles”) to a problem (“needing to be able to set variables on a site specific basis”) ---
**Author:** ![Chris\_Bennett](https://avatars.discourse-cdn.com/v4/letter/c/e19adc/32.png) [@Chris\_Bennett](https://forum.ansible.com/u/Chris_Bennett)\ **Post date:** [June 20, 2013, 5:37am UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/8 "2013-06-20T05:37:05Z")
Hi Michael, Thanks for your reply. > I tend to understand things better when we talk in terms of use cases > rather than syntax, I guess. > > If you need different variables for different sites, this is exactly what > "group\_vars/\" is for, so this may be a question of trying to > apply the wrong solution ("roles") to a problem ("needing to be able to set > variables on a site specific basis") When I say sites, I really meant customers. I have a project structure per customer, that exists in a per-customer directory. Things like the anisble config file for that customer's hosts, customer variables etc are stored here. I then have reusable roles that I want to store centrally without duplicating it per customer project structure. I'd like to avoid having separate customers variable files co-existing under some point in one project structure tree. Does that explain my use case more clearly? Thanks, Chris ---
**Author:** ![Jurgen\_Haas1](https://avatars.discourse-cdn.com/v4/letter/j/439d5e/32.png) [@Jurgen\_Haas1](https://forum.ansible.com/u/Jurgen_Haas1)\ **Post date:** [June 20, 2013, 7:08am UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/9 "2013-06-20T07:08:41Z")
I second that use case. We are in the same boat: As a service provider, we introduce our own best practices to all different customer projects and we want to migrate all projects to Ansible. Now, as an example, there is the role “apache” which contains everything to install and configure the Apache server which of course is driven by variables to cope with customer specific setups. But the general tasks, handlers, templates, etc. are all the same. What I was thinking is that we could have everything in one single directory tree and handle the customer specific stuff by different inventories. That would already work apart from the fact that developers and admins from the customer get involved in the maintenance of the playbooks and that would mean that they get to see everything from our other customers too - which is not a good idea. Hope this use case describes the requirement properly enough so that other experienced Ansible users can suggest adjustments or completely new ways. ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [June 20, 2013, 3:56pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/10 "2013-06-20T15:56:14Z")
What you want here is group\_vars/. Put all machines belonging to that one customer in a single group, and a different customer can set different variables. ---
**Author:** ![Jurgen\_Haas2](https://avatars.discourse-cdn.com/v4/letter/j/f0a364/32.png) [@Jurgen\_Haas2](https://forum.ansible.com/u/Jurgen_Haas2)\ **Post date:** [June 20, 2013, 4:24pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/11 "2013-06-20T16:24:01Z")
That is only one part of it. Yes, grouping hosts by customer makes a lot of sense. However, the whole directory tree with many dozen files from roles, variables and files on my own developer machine contains the superset of everything across all my customers. I want to share the relevant files for each customer with them, but only what belongs to them on not what belongs to others. ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [June 20, 2013, 4:29pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/12 "2013-06-20T16:29:59Z")
Sure, this is easy to do with multiple repos. You can specify roles dirs outside the repo if you want: roles: - { role: /path/to/other/role, … } - { role: some\_role\_that\_is\_assumed\_to\_be\_in\_roles\_subdirectory, … } And the same with vars\_files and normal task includes. So having a repo per customer, I think, is what you’d want to do. Each repo could include the group\_vars in their repo alongside the playbook if on 1.2+ and then could path the “internal” stuff using specified paths. This is the same way some folks like to handle secret information like passwords while also open-sourcing or sharing their main repo. ---
**Author:** ![Jurgen\_Haas2](https://avatars.discourse-cdn.com/v4/letter/j/f0a364/32.png) [@Jurgen\_Haas2](https://forum.ansible.com/u/Jurgen_Haas2)\ **Post date:** [June 20, 2013, 4:49pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/13 "2013-06-20T16:49:16Z")
Sounds very promising, I just may have one missing piece. Let’s have a look at an example: /somepath/ansible/ ./common/ ./group\_vars/ ./host\_vars/ ./roles/ /apache/ /mysql/ /security/ ./master.yml ./customer1/ ./group\_vars/ ./host\_vars/ ./roles/ ./hosts ./customer2/ ./group\_vars/ ./host\_vars/ ./roles/ ./hosts So, each customer has their own hosts file with the inventory. And they all have their group\_vars and host\_vars directory for their specific variables. Question: In the structure above i have group\_vars and host\_vars relative to the hosts file and relative to the master.yml playbook. How do I get ansible-playbook to first read the variables from below the master.yml and then potentially overwrite them from below the inventory? If that was possible I guess it would already be the perfect solution. ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [June 20, 2013, 4:54pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/14 "2013-06-20T16:54:00Z")
The group\_vars/host\_vars stuff can only go alongside the top level playbook AND the inventory file directory. You probably want to use vars\_files or roles instead, to effectively do something like the same thing. Just path the role or vars\_files you need to load in. ---
**Author:** ![martin\_f\_krafft](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@martin\_f\_krafft](https://forum.ansible.com/u/martin_f_krafft)\ **Post date:** [June 20, 2013, 5:10pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/15 "2013-06-20T17:10:02Z")
also sprach Michael DeHaan \ [2013.06.20.1829 +0200]: > roles: >    - { role: /path/to/other/role, ... } Nice, I didn't know this was possible. Would you consider extending this to allow for relative paths? Right now, the behaviour is a bit weird:   fishbowl:…a/ansible|master|playbooks% cat roles/motd/playbook.yml   - name: manage /etc/motd     hosts: motd\_hosts:!down     sudo: yes     roles:       - { role: .. }   fishbowl:…a/ansible|master|playbooks% grep name: roles/motd/tasks/main.yml   - name: write the motd file   - name: remove old motd.tail file   fishbowl:…a/ansible|master|playbooks% ansible-playbook roles/motd/playbook.yml --list-tasks   playbook: roles/motd/playbook.yml     play #1 (manage /etc/motd): task count=5       write the motd file       remove old motd.tail file The weirdness is in the role:.. (two dots, parent directory) statement. What I want is a way to use a role \*in the same directory\* as the playbook. However, '.' does not work ("cannot find role in …/roles/motd/roles"). I don't know why it looks for a directory ./roles/. Using two dots '..' works, but I cannot explain why. I'd like to use this functionality and would like this behaviour to be well-defined. Is that possible? Cheers, ---
**Author:** ![Jurgen\_Haas2](https://avatars.discourse-cdn.com/v4/letter/j/f0a364/32.png) [@Jurgen\_Haas2](https://forum.ansible.com/u/Jurgen_Haas2)\ **Post date:** [June 20, 2013, 7:18pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/16 "2013-06-20T19:18:43Z")
Tried this and can’t confirm yet. I have a group\_vars/all.yml file anlongside the the the top level playbook AND alongside the inventory file and only the variables from alongside the inventory file are known. From the example above I’m calling **ansible-playbook -i …/customer1/hosts master.yml** from within the master.yml directory but only variables from …/customer1/group\_vars/all.yml are known. I’m missing the variables from ./group\_vars/all.yml ---
**Author:** ![Jurgen\_Haas2](https://avatars.discourse-cdn.com/v4/letter/j/f0a364/32.png) [@Jurgen\_Haas2](https://forum.ansible.com/u/Jurgen_Haas2)\ **Post date:** [June 20, 2013, 7:28pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/17 "2013-06-20T19:28:12Z")
And even worse, it looks as if variables are resolved differently. The playbook looks like this: - hosts: “{{ myhost }}” connection: local gather\_facts: no tasks: - name: Test debug: msg=“Value of test is {{ test }} or $test and xyz is {{ xyz }} or $xyz” When I call **ansible-playbook master.yml --extra-vars “myhost=all” -i hosts** I gate this output: “ **Value of test is 1 or 1 and xyz is 23 or 23** ” When I call **ansible-playbook master.yml --extra-vars “myhost=all” -i …/customer1/hosts** I gate this output: “ **Value of test is {{test}} or 2 and xyz is {{xyz}} or $xyz** ” ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [June 20, 2013, 8:24pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/18 "2013-06-20T20:24:52Z")
Already supports relative paths. ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [June 20, 2013, 8:26pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/19 "2013-06-20T20:26:50Z")
The group\_vars all location relative to the playbook must be relative to THE playbook being executed. This is why all top level playbooks in ansible-examples live where they do. I think it would be confusing if you had to store them in more than one place. In your example, I’d recommend storing playbooks at top level OR keeping them with your inventory. ---
**Author:** ![Chris\_Bennett](https://avatars.discourse-cdn.com/v4/letter/c/e19adc/32.png) [@Chris\_Bennett](https://forum.ansible.com/u/Chris_Bennett)\ **Post date:** [June 20, 2013, 11:26pm UTC](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990/20 "2013-06-20T23:26:14Z")
Hi Michael, > roles: >    - { role: /path/to/other/role, ... } >    - { role: some\_role\_that\_is\_assumed\_to\_be\_in\_roles\_subdirectory, ... } Excellent, this works well for my use case. I'd like to avoid duplicate pathing for a location that has common roles though. roles:   - { role: /path/to/other/role1, ... }   - { role: /path/to/other/role2, ... } If there was an environment variable or playbook variable that was observed in role location lookup, this could be managed without the playbook having static paths. Does variable subst work in the { role: [..] } section? My tests with:   vars:     role\_base\_path: /path/to/other/   roles:     # any of     - { role: ${role\_base\_path}/role1 }     - { role: $role\_base\_path/role1 }     - { role: {{ role\_base\_path }}/role1 } all failed with varying errors. If there was some form of role search path & precedence behaviour, you could have one or more common role locations, plus the relative roles/ path, and simplify the playbook to   roles:     - webserver Overriding roles in the relative roles path might also be useful for testing & development as you could just copy over & modify the relative role & not have to change the playbook. Thanks for your help so far on this - I don't really think I'm over-complicating things, being too clever, or programmatic, but if you think this use case doesn't fit with ansible at this point in time, I'll make do with the duplication and watch closely & hope it comes in the future 🙂 Regards, Chris [Next page](https://forum.ansible.com/t/multi-role-organization-and-re-use/12990.md?page=2)