# How to arrange structure for the case multiple projects **URL:** \ **Category:** Get Help\ **Tags:** getting-started\ **Created:** [November 14, 2023, 9:28am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355 "2023-11-14T09:28:43Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![lanhnguyen](https://avatars.discourse-cdn.com/v4/letter/l/5fc32e/32.png) [@lanhnguyen](https://forum.ansible.com/u/lanhnguyen)\ **Post date:** [November 14, 2023, 9:28am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/1 "2023-11-14T09:28:43Z")
If I work with many projects, each project we use different role. Are there any good ways to arrange structure for this case? ---
**Author:** ![utoddl](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/utoddl/32/249_2.png) [@utoddl](https://forum.ansible.com/u/utoddl)\ **Post date:** [November 14, 2023, 7:06pm UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/2 "2023-11-14T19:06:00Z")
Hi @lanhnguyen , and welcome to the Ansible forum! Thanks for asking this question, because I’ve recently started to question how we’ve been arranging our projects for the last 7 years. It’s been working for us, but maybe that’s because we started with some fundamentally wrong assumptions. Anyway, ignoring what’s weird about _our_ practices, here’s the “normal” part of how we’ve arranged our projects. We’re responsible for about two dozen “service lines”, and for each of those we have one Ansible “project”. And by “project” I mean a git repo with a corresponding “SCM project” (that word is over-used!) in our own GitLab instance, as well as one (usually one, but sometimes more, and sometimes none) “AWX project(s)” (see?) in our AWX instance. These “service line projects” (git repositories) typically have a dozen or so Ansible playbooks at the top, as well as a “roles” directory that contains roles specific to that service line. We rarely have any task files, templates, plugins, etc. at the project level; such assets are almost always part of a role. Inside the “role” directory we also have one of the two “requirements.yml” files; the other one is in the top-level “collections” directory, which also contains “ansible\_collections/mw/\/”. The latter contains any local, project-specific collections; we’re unlikely to create any new local roles outside of these local collections. Besides these “service line” projects, we also have developed a handful of what we call “mw\_common\_\*” roles. These include roles to install/configure: - locally-built packages, - apache web servers to our group standards and expectations, - tomcat / wildfly services - log rotation - monitoring These are all listed as required in each service-line project’s “roles/requirements.yml” file. Similarly, collections that aren’t local to a service-line project are listed in the “collections/requirements.yml” file. Production jobs for all this happen in our AWX instance. But development/testing work can happen either on our primary workstations or on our “bastion” hosts. Typically any of the projects we’re interested in are "git clone"d side-by-side in some work directory of our choosing. We have about 8 worker bees in our “Middleware” group (hence the “mw” popping up in our project names), so we have about 8 different editors in play. (Our “my editor is better than your editor” _discussions_ are always fun!) Besides these service-line projects and `mw_common_*` projects, we have a few more. One is a skeleton project that we use as a template for new projects. Another, “defaults,” contains our agreed upon `ansible-lint` config, some “extremely global” variables that all projects incorporate, and our group-wide static inventory which works for us because we have almost all static hosts. Plus there are a few scripts for housekeeping duties like running `ansible-galaxy` commands when needed. Finally, certain operations on our GitLab projects trigger Jenkins jobs via web hooks, like linting, and causing SCM updates in AWX when relevant branches are updated. And that’s about it. Maybe something in here answers your question, or inspires more questions. I hope so. ---
**Author:** ![lanhnguyen](https://avatars.discourse-cdn.com/v4/letter/l/5fc32e/32.png) [@lanhnguyen](https://forum.ansible.com/u/lanhnguyen)\ **Post date:** [November 15, 2023, 1:15am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/3 "2023-11-15T01:15:14Z")
> [@utoddl](#): > > These are all listed as required in each service-line project’s “roles/requirements.yml” file. Similarly, collections that aren’t local to a service-line project are listed in the “collections/requirements.yml” file. So the content in “roles/requirements.yml” and “collections/requirements.yml” are same? ---
**Author:** ![lanhnguyen](https://avatars.discourse-cdn.com/v4/letter/l/5fc32e/32.png) [@lanhnguyen](https://forum.ansible.com/u/lanhnguyen)\ **Post date:** [November 15, 2023, 1:18am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/4 "2023-11-15T01:18:53Z")
> [@utoddl](#): > > We’re responsible for about two dozen “service lines”, and for each of those we have one Ansible “project”. And by “project” I mean a git repo with a corresponding “SCM project” (that word is over-used!) in our own GitLab instance, as well as one (usually one, but sometimes more, and sometimes none) “AWX project(s)” (see?) in our AWX instance. These “service line projects” (git repositories) typically have a dozen or so Ansible playbooks at the top, as well as a “roles” directory that contains roles specific to that service line. We rarely have any task files, templates, plugins, etc. at the project level; such assets are almost always part of a role. Inside the “role” directory we also have one of the two “requirements.yml” files; the other one is in the top-level “collections” directory, which also contains “ansible\_collections/mw/\/”. The latter contains any local, project-specific collections; we’re unlikely to create any new local roles outside of these local collections. For example, I have project1 and project2 and project3. I need to install httpd, php, mysql for project 1, and install nodejs and mongodb for project 2, and install ngx, php, mariadb for project 3. For your idea, I create each ansible folder for each project, right? In this case how do I run playbook file for each project on the server that ansible is installed or on the local machine? current I’m using Windows 11 Pro which has ubuntu 20.04.6 LTS APP. and I install ansible into ubuntu 20.04.6 LTS APP. If I create an ansible folder on projects, these projects are on a folder of windows. How do I run playbook files for each project from ubuntu 20.04.6 LTS APP? ---
**Author:** ![utoddl](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/utoddl/32/249_2.png) [@utoddl](https://forum.ansible.com/u/utoddl)\ **Post date:** [November 15, 2023, 5:40am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/5 "2023-11-15T05:40:02Z")
> [@lanhnguyen](#): > > So the content in “roles/requirements.yml” and “collections/requirements.yml” are same? No, they are different. Confusion is understandable though, especially if you’re coming to Ansible at this “interesting time.” There are these roles in the `roles/` directory, but there are also collections which may or may not contain roles… Why is it split up this way? Ansible has evolved _a lot_ in the last few years. Before _collections_ became a thing, _roles_ were the only sharable units of work. The `roles/requirements.yml` file allowed you to pull shared resources into a project in the form of individual roles. That worked okay-ish for simple stand-alone roles, but complex solutions couldn’t be easily stuffed into such a simple box. For example, suppose a network vendor published a bunch of roles that work together to configure their suite of networking equipment. Most of those roles use some special custom plugins. Which of those roles is the right one to contain one of those plugins? Well, because any customer may only have a small fraction of the vendor’s devices, they may only use the small subset of those roles that they need to configure the devices they own. So each of those roles will have to contain a copy of _all the same plugins._ Only, they aren’t all the same, because the vendor doesn’t do a great job of keeping them in sync, or the customers only upgrade individual roles when they hit a problem so they have different possibly conflicting versions installed, or whatever. But in the end every customer has to do version management on a set of roles they didn’t write, all because, fundamentally, individual roles provide an inadequate unit for sharing work _at scale_. What we need is a sharable unit where cross-role redundancies can be eliminated, and cross-role dependencies can be collected, tested, managed, and maintained as a _collection_. And thus _collections_ were born. A collection can contain any number of roles - even zero. Unlike individually distributed roles, if the roles in a collection all need some plugin, they can share a single copy. A collection can be tested and distributed as a unit, so users don’t have to track down the right version of every asset. All the pieces of a collection should work together. - The old style of distributing roles still works, though. Thus, the older simple shared roles that we wrote years ago and still use are listed in projects’ `roles/requirements.yml` files. - Newer work - shared roles and associated plugins - have been put into collections, so they are listed in projects’ `collections/requirements.yml` files. - Roles and collections that aren’t distributed or shared among projects, but are instead specific to a project can live just fine in that project’s `roles/` and `collections/ansible_collections/` directories. They don’t need to be listed in any `requirements.yml` files. ---
**Author:** ![utoddl](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/utoddl/32/249_2.png) [@utoddl](https://forum.ansible.com/u/utoddl)\ **Post date:** [November 15, 2023, 6:37am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/6 "2023-11-15T06:37:54Z")
| | | | | | --- | --- | --- | --- | | **project1** : | httpd | php | mysql | | **project2** : | nodejs | mongodb | | | **project3** : | ngx | php | mariadb | There are no hard and fast rules, but in our shop each of `project1`, `project2`, and `project3` would be its own git repo / Ansible project. Say we do `project1` first. We’d end up with three roles: `roles/httpd/`, `roles/php/`, and `roles/mysql/`. Skip `project2` for now and move on to `project3`. `httpd` and `ngx` are both web servers, but not very similar in their configs, so it’s probably more work than it’s worth to abstract away their differences in some common role. So we’ll end up with a `role/ngx/`. However, both `project1` and `project3` need `php`, so we’d probably turn `project1`’s `role/php/` into a shared role (so _it_ becomes its own git repo / Ansible project). Likewise, `mysql` and `mariadb` are similar enough, or used to be, that we could probably turn `project1`’s `role/mysql/` into a shared `mymariasqldb` role (just kidding about that name). Back to `project2`, `nodejs` and `mongodb` probably end up being their own roles in that project. Any of these could be turned into shared roles later if another project comes along that needs one of them. I don’t understand your question about Windows. But it’s 1:30am local, and the last Windows I ran was WinME, so I’m not your best source for that info. Sorry. ---
**Author:** ![lanhnguyen](https://avatars.discourse-cdn.com/v4/letter/l/5fc32e/32.png) [@lanhnguyen](https://forum.ansible.com/u/lanhnguyen)\ **Post date:** [November 15, 2023, 7:01am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/7 "2023-11-15T07:01:30Z")
> [@utoddl](#): > > I don’t understand your question about Windows. But it’s 1:30am local, and the last Windows I ran was WinME, so I’m not your best source for that info. Sorry. I mean I’m using windows os not linux or ubuntu machine. On project1 folder, I create application folder and ansible folder. application folder contains source code. ansible folder contains ansible files like as host, playbook, roles… project 2 and and project 3 are same. these folder project1, project2, project3 are in folder of the windows os. But to run playbook file , I run in ubuntu 20.04.6 LTS APP ![image](https://us1.discourse-cdn.com/ansible/original/2X/2/23980cd385f530236381c3384b134235c94b3864.png) how do I run the playbook file in the folder project1 project2 project3? ---
**Author:** ![ptn](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/ptn/32/481_2.png) [@ptn](https://forum.ansible.com/u/ptn)\ **Post date:** [November 15, 2023, 9:00am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/8 "2023-11-15T09:00:03Z")
Hi, That’s called [WSL](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux) FYI. And to answer your question: You should be able to access Windows filesystem from your WSL machine; IIRC, Windows C: drive is mounted on /mnt/c. Another way to do it would be to use git; put your files in a repo and `git clone` where you need to work on / use your files. Though having two or more clones lying around on your machine could become untidy. I suggest you use git to version your files, then clone once wherever you want and work from the same path on both machines (Windows, WSL). ---
**Author:** ![chris](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/chris/32/293_2.png) [@chris](https://forum.ansible.com/u/chris)\ **Post date:** [November 15, 2023, 12:38pm UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/9 "2023-11-15T12:38:30Z")
This is exactly what we have, with the roles specifically written so they can be open and shared while the project repos are private, FWIW I have written roles for almost these applications, some more complex and up to date than others, some not very polished but all GPL’d: - [webarch / apache · GitLab](https://git.coop/webarch/apache) - [webarch / mariadb · GitLab](https://git.coop/webarch/mariadb) - [webarch / mongodb · GitLab](https://git.coop/webarch/mongodb) - [webarch / nginx · GitLab](https://git.coop/webarch/nginx) - [webarch / nodejs · GitLab](https://git.coop/webarch/nodejs) - [webarch / php · GitLab](https://git.coop/webarch/php) And here is a (rather messy!) example repo that uses them to build a dev server: - [webarch / wsh · GitLab](https://git.coop/webarch/wsh) ---
**Author:** ![lanhnguyen](https://avatars.discourse-cdn.com/v4/letter/l/5fc32e/32.png) [@lanhnguyen](https://forum.ansible.com/u/lanhnguyen)\ **Post date:** [November 16, 2023, 9:26am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/10 "2023-11-16T09:26:09Z")
> [@ptn](#): > > And to answer your question: You should be able to access Windows filesystem from your WSL machine; IIRC, Windows C: drive is mounted on /mnt/c. > Another way to do it would be to use git; put your files in a repo and `git clone` where you need to work on / use your files. Though having two or more clones lying around on your machine could become untidy. > > I suggest you use git to version your files, then clone once wherever you want and work from the same path on both machines (Windows, WSL). Thanks for your idea so much. Do you think this is the good way for managing multiple project with using ansible to deploy? ```auto Project1 application source . . . ansible playbook.yaml hosts ansible.cfg roles/ common/ tasks/ main.yml handlers/ main.yml templates/ ntp.conf.j2 files/ bar.txt foo.sh vars/ main.yml defaults/ main.yml meta/ main.yml Project2 application source . . . ansible playbook.yaml hosts ansible.cfg roles/ common/ tasks/ main.yml handlers/ main.yml templates/ ntp.conf.j2 files/ bar.txt foo.sh vars/ main.yml defaults/ main.yml meta/ main.yml ``` Do you have any good way to arrange structure of ansible for many projects ---
**Author:** ![ptn](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/ptn/32/481_2.png) [@ptn](https://forum.ansible.com/u/ptn)\ **Post date:** [November 20, 2023, 6:23pm UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/11 "2023-11-20T18:23:13Z")
> [@lanhnguyen](#): > > Do you think this is the good way for managing multiple project with using ansible to deploy? Is ‘roles’ folder on each project the roles install path or would it effectively be committed to the same repo as your playbook.s, inventor.y.ies and config files ? Generally (in case you don’t write collections that is), each role has its own repo. Then you install each of them you need (be it with `ansible-galaxy` using a requirements.yml file or not, direct git clone, or else) to a location of your choosing (either by defining roles install paths in your config or using `ansible-galaxy -p` parameter) for Ansible to find and use when you run your playbooks. Then each of your project would only need to have its playbook.s, inventor.y.ies, requirements.yml file.s, config files, … Here is how I usually do it: - One role per repo (same structure as @chris provided examples) - Either one repo per project (mostly deployment projects linked to a specific project -as in thing we build-), or one repo containing multiple ‘projects’, mostly for configuration management; here is what I mean: ---
**Author:** ![lanhnguyen](https://avatars.discourse-cdn.com/v4/letter/l/5fc32e/32.png) [@lanhnguyen](https://forum.ansible.com/u/lanhnguyen)\ **Post date:** [November 21, 2023, 1:22am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/12 "2023-11-21T01:22:57Z")
> [@ptn](#): > > Here is how I usually do it: > > - One role per repo (same structure as @chris provided examples) > - Either one repo per project (mostly deployment projects linked to a specific project -as in thing we build-), or one repo containing multiple ‘projects’, mostly for configuration management; here is what I mean: @ptn We don’t put project’s source code and ansible in same repo? ---
**Author:** ![lanhnguyen](https://avatars.discourse-cdn.com/v4/letter/l/5fc32e/32.png) [@lanhnguyen](https://forum.ansible.com/u/lanhnguyen)\ **Post date:** [November 21, 2023, 2:07am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/13 "2023-11-21T02:07:49Z")
> [@ptn](#): > > ```auto > requirements.yml > > ``` @ptn Execute me, What we write in requirements.yml file ? ---
**Author:** ![lanhnguyen](https://avatars.discourse-cdn.com/v4/letter/l/5fc32e/32.png) [@lanhnguyen](https://forum.ansible.com/u/lanhnguyen)\ **Post date:** [November 21, 2023, 2:12am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/14 "2023-11-21T02:12:18Z")
> [@ptn](#): > > One role per repo (same structure as @chris provided examples) In the case each project we need to clone difference source code from github and run some commands. For example after clone laravel source we need to move to the folder that contains source to run the commands like as composer install, php artisan optimize, php artisan migrate… or another project, we need to clone nodejs source them, after clone we run the command npm install… So Where we push role for this part of each project? ---
**Author:** ![ptn](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/ptn/32/481_2.png) [@ptn](https://forum.ansible.com/u/ptn)\ **Post date:** [November 21, 2023, 12:24pm UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/15 "2023-11-21T12:24:17Z")
> We don’t put project’s source code and ansible in same repo? Depends. But roles should be in their own repo. You can picture a role as reusable content, like a library, and most of the time, it doesn’t make sense to have this kind of shared / reusable content in the same spot your project resides. Please also note that `ansible-playbook` only needs to know where your roles are on your filesystem; it doesn’t care about where it comes from and how it got there. **But** if you use `ansible-galaxy` to install your roles (= creates an archive from your repo, be it your SCM or a Galaxy repo, then unarchive its content on your ansible config defined roles path (overridable)), it won’t be able to if your repo structure is not the expected one, which usually means one role per repo. I think you could technically do some magic in a parent meta/main.yml file to share a repo between roles, but AFAIK, it’s not the intended way to do so; for this usecase, better use a collection. Now your question is a bit vague; if you are talking about playbooks, inventories, host/group vars or ansible.cfg files, then yes, you could put them next to your project files. It depends on your needs, really. For instance, having a playbook that build or deploy your project in the same repo is totally fine. And just to be clear: There is no playbook in role, only yaml files defining resources. You can’t use a role by itself, it has to be imported/included from a playbook. Hope it helps to clear things up. > What we write in requirements.yml file ? The requirements 😛. Joke aside, check this out: [Galaxy User Guide — Ansible Documentation](https://docs.ansible.com/ansible/latest/galaxy/user_guide.html#installing-multiple-roles-from-a-file). The idea is to list in a declarative way the resources to install. These are the resources (here collections and/or roles) you’ll need for your tasks, like a custom collection or role that is not shipped with ansible package. ---
**Author:** ![utoddl](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/utoddl/32/249_2.png) [@utoddl](https://forum.ansible.com/u/utoddl)\ **Post date:** [November 21, 2023, 11:07pm UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/16 "2023-11-21T23:07:24Z")
> [@ptn](#): > > But roles should be in their own repo. That’s a rather broad characterization. It’s certainly possible to have project-specific roles that will never be shared with other projects. We have one mega-project that has 49 local roles, almost none of which even makes sense outside of the context of that project. Splitting it up into internal roles like that gives an organizing framework without which the [_goes to count…_] 858 files therein (2979 if you include group\_vars and host\_vars) would be unmanageable. Also, there’s some overhead with each step of moving assets through “local files"→"local roles"→"shared roles"→"collections”. You gain something with each step also, but whether increasing the footprint is worth it is a judgement call every time. (I have a pet argument from my early days of learning Ansible that there should be a way to roll up an entire project into a single `.yaml` file, so, going off the left edge of the little map up there. The proliferation of files still bothers me, but I digress.) My point, if I have one, is that there are lots of ways to do it that will work. What’s best is up to whoever has to maintain it. And it’s okay to change your mind about that later if you want to try it a different way. Otherwise, yeah, @ptn is giving you good stuff to think about. ---
**Author:** ![lanhnguyen](https://avatars.discourse-cdn.com/v4/letter/l/5fc32e/32.png) [@lanhnguyen](https://forum.ansible.com/u/lanhnguyen)\ **Post date:** [November 22, 2023, 4:24am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/17 "2023-11-22T04:24:59Z")
> [@chris](#): > > And here is a (rather messy!) example repo that uses them to build a dev server: > > - [webarch / wsh · GitLab](https://git.coop/webarch/wsh) @chris In users.yaml file , roles is users, where do you define this role? ---
**Author:** ![chris](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/chris/32/293_2.png) [@chris](https://forum.ansible.com/u/chris)\ **Post date:** [November 22, 2023, 9:57am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/18 "2023-11-22T09:57:02Z")
In `requirements.yml`: ```yaml - name: users src: https://git.coop/webarch/users.git version: master # users scm: git ``` And `ansible.cfg` specifies where it’ll be checked out to: ```ini [defaults] roles_path = galaxy/roles ``` ---
**Author:** ![lanhnguyen](https://avatars.discourse-cdn.com/v4/letter/l/5fc32e/32.png) [@lanhnguyen](https://forum.ansible.com/u/lanhnguyen)\ **Post date:** [November 23, 2023, 2:00am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/19 "2023-11-23T02:00:01Z")
@chris I did same you, I defined roles\_path in config file. ![image](https://us1.discourse-cdn.com/ansible/original/2X/4/404fa39f569b4eca8c7d087eceb2146d9caecdcf.png) ![image](https://us1.discourse-cdn.com/ansible/original/2X/2/2f230a076562fa17744ae11f493e31f457461f80.png) The I run the command to pull role source from github but it didn’t pull. Do you know why? ![image](https://us1.discourse-cdn.com/ansible/original/2X/7/731565816f3edc31cc19ea57a05704a9fdbf3b58.png) I used your git but It also can not pull ![image](https://us1.discourse-cdn.com/ansible/original/2X/b/bf67944cf2938859fc18103437742ea5a00a31af.png) ---
**Author:** ![lanhnguyen](https://avatars.discourse-cdn.com/v4/letter/l/5fc32e/32.png) [@lanhnguyen](https://forum.ansible.com/u/lanhnguyen)\ **Post date:** [November 23, 2023, 2:40am UTC](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355/20 "2023-11-23T02:40:18Z")
It always load config file from /etc/ansible/ansible.cfg. How do we set config file in another path? [Next page](https://forum.ansible.com/t/how-to-arrange-structure-for-the-case-multiple-projects/2355.md?page=2)