Hi,
I’m looking into using Ansible to manage my systems and I’m wondering about how to make my roles/tasks composable.
As far as I understand roles are Ansibles version of reusable components but it seems they do not really support any sort of ordering/dependencies. Let me give an example of what I’m trying to accomplish:
I download a role “mysql-server” from github that installs the mysql server package and does some initial configuration, etc.
When setting up a system I also need to
a) install a repo before the mysql-server role
b) do some customization to mysql after the mysql-server role (add users, etc.)
c) apply the “register-server” role which performs various tasks to mark the server as ready for work
From what I understand there is no way to mix roles and tasks which makes me wonder how to implement such self-contained roles that can be used as if they were regular tasks? I guess what I’m asking for is some sort of “apply_role” action that would make it possible to build more complex playbooks out of self-contained components.
Is this already possible and I just missed it or is there something wrong with what I’m trying to do?
From a roles/foo/tasks/main.yml you can include other task files.
How do I do this without resorting to absolute paths?
I tried “- include: mysql-server” but this obviously looks for mysql-server in the tasks directory for the current role.
Further, you can include a loose task section with roles just fine.
Hi,
I'm looking into using Ansible to manage my systems and I'm wondering
about how to make my roles/tasks composable.
As far as I understand roles are Ansibles version of reusable components
but it seems they do not really support any sort of ordering/dependencies.
Let me give an example of what I'm trying to accomplish:
- I download a role "mysql-server" from github that installs the mysql
server package and does some initial configuration, etc.
- When setting up a system I also need
a) install a repo before the mysql-server role
b) do some customization to mysql after the mysql-server role (add
users, etc.)ou'll
c) apply the "register-server" role which performs various tasks to mark
the server as ready for work
From what I understand there is no way to mix roles and tasks which makes
me wonder how to implement such self-contained roles that can be used as if
they were regular tasks? I guess what I'm asking for is some sort of
"apply_role" action that would make it possible to build more complex
playbooks out of self-contained components.
Is this already possible and I just missed it or is there something wrong
with what I'm trying to do?
I believe you can do this because we do a similar thing.
We define reusable tasks in roles and include them in a playbook. ex
roles:
- yumrepos
- mysql-server
If you would like to have seperate tasks that don't fit in this reusable
role model you can run pre_tasks and post_tasks in the playbook (you'll
need a rather recent 1.2 version)
ex :
pre_tasks:
- name: do something
action: command "run this command"
roles:
- yumrepos
- mysql-server
post_tasks:
- name: create DB users
action: mysql ......
These get executed in the mentioned order as far as i know.
The problem is that this only works in very specific ways and is no general solution to making things truly reusable.
I understand that using this approach can be used for coarse grained:
task
role(s)
task
but what if I have more complex compositions like:
task
role
task
role
task
…
For this the pre/post approach is not usable.
What I’m really getting at is that roles should be first class citizens and be usable anywhere an action can be used.
The reason I’m now looking at Ansible is because I ran into this problem with Puppet when trying to deploying OpenStack. In Puppet this “containment issue” is a known issue and even has a page about it here: http://docs.puppetlabs.com/puppet/3/reference/lang_containment.html
I wanted to reuse the existing openstack::controller class from stackforge but I needed to define a yum repo with the grizzly packages before that class was applied. Since dependencies stop working when you begin to compose classes like this that didn’t work out and that is a major issue for me.
Michael mentioned above that what I’m looking for should be possible but I’m not sure how the syntax for this would look.
The problem is that this only works in very specific ways and is no
general solution to making things truly reusable.
I understand that using this approach can be used for coarse grained:
These generalizations are simply not the case.
- task
- role(s)
- task
but what if I have more complex compositions like:
- task
- role
- task
- role
- task
- ...
Roles *are* just automations around grouping tasks in a folder structure
where handlers and variables can also live in that structure. You can
ABSOLUTELY only use roles.
For this the pre/post approach is not usable.
It's totally usable.
pre/post tasks are there to encourage handler runs to flush for load
balancing and other tasks.
What I'm really getting at is that roles should be first class citizens
and be usable anywhere an action can be used.
Roles are task includes.
They are basically exactly the same thing.
You can never do a "tasks:" foo and you will be perfectly fine.
If you want tasks to include other statements, it is as simple as:
I think rather than discussions of ‘composability’ it’s really saying “this role should be able to designate that it requires this other role when it is loaded”
That would probably be doable.
Not an immediate priority but I have some ideas on syntax.
I did mention that this might already be possible and I also mentioned what I wanted to accomplish in my initial posting so I’m not sure why you claim I didn’t?
The “not possible” only referred to interleaving roles and tasks though that might have more to do with my lack of understanding of the syntax.
What I am hoping for is that roles are treated in a similar way to modules. For modules there is a search path and I can reference a module without having to specify where they live no matter from where I reference them. When I use “action: ” then Ansible finds in the module library. What I would like to be able to do is to use something like “role: mysql_server” and then have Ansible look up the module in the “role library” and include /mysql_server/tasks/main.yaml.
That way people could create roles on github and users could git-clone these into the roles directory and then reference these no matter where the playbook they execute lives relative to that role.
The resulting playbook for my example would then look like this:
name: configure repo for patched mysql server
copy: src=my-repo.repo dest=/etc/yum.repos.d/my-repo.repo
name: install and prepare mysql
role: mysql-server root_password=test
name: add additional mysql user
mysql_user: name=admin host=‘%’ …
name: register this database server with several external services
role: register_db_server
Here roles are basically used like modules and that is what I meant when I said they should become first class citizens.
But there are not four tasks but two tasks and two roles interleaved and the roles “mysql_server” and “register_db_server” are not written by me but by somebody else and git-pulled into the roles directory. I only want to change these for bug fixes/features I want to contribute to upstream and pull updated versions but not for customizations. These customizations would happen in a higher level role/playbook like the one in my example.
Later I might want put this higher level role on github for other users to use and extend in the same way.
If I understand the things explained to me so far correctly about syntax and features in Ansible than what comes closest to this is the following version of the example above:
name: configure repo for patched mysql server
copy: src=my-repo.repo dest=/etc/yum.repos.d/my-repo.repo
name: add additional mysql user
mysql_user: name=admin host=‘%’ …
include: …/…/register_db_server/tasks/main.yml
This should work but but has two issues:
Having to specify the fill path to the main.yml of the role is somewhat ugly when I really want to say “apply role X”. This is not that big a deal but so far I find Ansibles “language” rather elegant and concise and it would be more explicit if I could declare that I want to apply a role rather than a specific playbook in a directory that just happens to be a role.
When a dependency is missing (e.g. I forgot to download the “mysql_server” role) I get a “File not found” error for the main.yml file. If the inclusion of a role could be specified more explicitly than this could be turned into a “Role or File X not found” message.
So basically this is not so much about features but about improving the syntax for this.
Here are three ways this could potentially be solved. All of them would probably require the addition of a “roles_path = (::…)” option in ansible.cfg:
Make include not only look for specified files but also look if a role with the name exists and include “<roles_path>//tasks/main.yml” in that case.
The result would look like this:
include: mysql_server key=value
Ansible would see that a role “mysql_server” exists in and include “/mysql_server/tasks/main.yml”. If no role or file with that name exists the error would be “Role or file mysq_server not found”.
The downside of this is that technically a file that is literally called “mysql_server” could exist in the current directory so the reference could be ambiguous. Not sure how likely that is though.
Add the syntax “- role: mysql_server key=value” to the language that would internally simply call include “/mysql_server/tasks/main.yml”.
This would be more explicit and disambiguate the inclusion of playbook files vs. the inclusion of roles but it would also introduce a new element to the language which might not be desired.
With the following approach I’m not sure if this is even possible as it basically turns role inclusion into a module call and I’m not sure about whether “treat roles like modules” is even possible because of how playbooks and modules interact (I think Ansible calls certain hook function of the module which would not be possible with a role). So just ignore the following if it is too crazy.
Make a module invocation not only look for a module with the specified name but also for a role (similar to case 1. above):
name: setup mysql server
mysql_server: key=value
In this case Ansible would not find a module “mysql_server” but instead of aborting with an error would first look for a role called “mysql_server” and if that exists include that instead similar to the cases above. This basically turn roles into modules except that you don’t have to write an actual module in python/ruby/erlang.
One way of realizing such tasks/roles interleaving could be to separate it into multiple plays inside one playbook. For example I used to process everything in 4 plays:
Bootstrap play - only bootstraps Ansible (installs python and python-apt with raw module)
Basic play - configures basic system settings, such as repositories
Infrastructure play - installs and does basic configuration of services (eg. MySQL, Nginx…)
Application play - installs and configures the application
But after thinking about it and refactoring I only use Bootstrap, Basic/Infrastructure (only does what Basic used to do) and Application (only installs roles of applications that now contain requirements for services (what used to be in Infrastructure)). This is simply possible with the use of “meta: flush_handlers”. But real dependencies between roles would be much better.
The problem here is that you basically have to make assumptions about the structure of the roles you are using and that they only restrict themselves to one of the above phases. That works if you write all the roles yourself but the entire reason I brought this up is that I want to understand if it is possible to create a sort of repository of roles similar to puppetforge where users can contribute their roles for other people to use and collaborate on. Now such a repository doesn’t have to be centralized and people can simply put their roles on github but I think it would be useful to at least make it possible to properly encapsulate roles in a way so that you don’t have to make assumptions about where exactly they live or how they are structured.
Namespacing and dependecies are interesting topics too but for now I’m primarily interested in finding out if/how such components could be integrated and so far it seems that roles or pretty much capable of supporting this but the language doesn’t yet support something like this directly.
Saying you don’t like ansible and then suggesting people use something else and linking to it isn’t a good way to make friends. You could have left that part out.
I’ll help you anyway though, but please don’t do it again.
I strongly disagree that roles make any assumptions about whether they are for bootstrap/basic/infrastructure/application. You will see in the documentation and code there’s nothing in there at all about these kinds of things and they are quite general purpose.
As for a repository of roles, we’re working on that. Stay tuned!
Roles already support dependencies as well since 1.3.
Saying you don’t like ansible and then suggesting people use something else and linking to it isn’t a good way to make friends. You could have left that part out.
Was not the intention…
I have been using Ansible for ~a year and hit many roadblocks throughout the development.
I did add that you are doing a good job keeping up to pace on issues.
Roles already support dependencies as well since 1.3.
This does kinda solve the issue, however we want to treat the role as readonly because changing it could break other playbooks.
Anyway, keep up the good work and best of luck with AWX