Ansible Roles in 1.8.4 - Still inferior to include files for complex playbooks?

Archives Ansible Project
1

From Ansible doc:

Roles in Ansible build on the idea of include files and combine them to form clean, reusable abstractions

I really want to love roles and use them extensively in our system, but so far I’m getting a heap of frustration instead. I am trying to position Ansible at work as the preferred automation platform to manage a complex system with several internally developed apps, plus db, web server, and more. (together this is one product/system) Multiple environments too, as usual.

I’d think it makes sense to have roles like:

  • appX (for X in 1 through N)
  • web
  • db
  • etc (more… skipped)

One seemingly basic thing I need to do is to restart this entire system occasionally. I would like to use Ansible to do that. Here’s a simplified pseudo-code recipe of a playbook:

  • web.startMaintenance()
  • app1.stop()
  • app2.stop()
  • app2.start()
  • app1.start()
  • web.endMaintenance()

I want to run a single playbook command to do this. That seems clearly possible with include files but not possible with roles. The following doesn’t work:

ansible-playbook --tags “app1.stop,app1.start,app2.start,app2.stop,web.startMaintenance,web.endMaintenance” playbooks/testRoleDeps.yml

That doesn’t work because all plays in the playbook are filtered by the tags provided, and each play needs to have a particular tag applied (first stop then start later), not all of them.

My conclusion? As of 1.8.4, roles are inadequate for complex playbooks and include files need to be used instead.

That’s unfortunate for code reuse reasons, etc. I would love to hear that this is not the case and I have overlooked something. Maybe a feature to resolve this is in the release plan?

I can’t figure out how to do this even after reading the documentation repeatedly, reading the ansible O’Reilly book (pre-release copy), the code in ansible-examples, and various other resources.

2

Some replies inline!

3

Some replies inline!

Thanks for all the help! I'll reply inline too.

From Ansible doc
<http://docs.ansible.com/playbooks_roles.html#introduction&gt;:

Roles in Ansible build on the idea of include files and combine them to *form

clean, reusable abstractions*

I really want to love roles and use them extensively in our system, but
so far I'm getting a heap of frustration instead. I am trying to position
Ansible at work as the preferred automation platform to manage a complex
system with several internally developed apps, plus db, web server, and
more. (together this is one product/system) Multiple environments too, as
usual.

I'd think it makes sense to have roles like:
- appX (for X in 1 through N)
- web
- db
- etc (more.. skipped)

That's usually the case for simple stuff, but for each app or component of
the system, I usually end up with multiple roles. Maybe a role, a role to
configure, a role to restart. When you start wanting to call individual
parts of a single role separately, that's a good sign that you need to
break that role up into multiple parts. For organizational purposes, you
can definitely do:

roles/web/setup/*
roles/web/install/*
roles/web/config/*
roles/web/restart/*

This is very helpful. I have provided some sample code below showing how I
have used this to do what I intended.

One seemingly basic thing I need to do is to restart this entire system
occasionally. I would like to use Ansible to do that. Here's a simplified
pseudo-code recipe of a playbook:
- web.startMaintenance()
- app1.stop()
- app2.stop()
- app2.start()
- app1.start()
- web.endMaintenance()

*I want to run a single playbook command to do this.* That seems clearly
possible with include files but not possible with roles. The following
doesn't work:

ansible-playbook --tags
"app1.stop,app1.start,app2.start,app2.stop,web.startMaintenance,web.endMaintenance"
playbooks/testRoleDeps.yml

It seems like a single playbook for all of these different configurations

might not be realistic. Why not have multiple playbooks, one per general
operation, and call the roles as needed?

The entire mission of Ansible is "simple IT automation", right? :slight_smile: Wherever
possible, my preference would be for a single Ansible playbook to handle
the execution of a single high-level "goal"/"user request". If that isn't
the case, then one needs to somehow bundle those externally, typically in a
script. That requires the local "Ansible team" to decide on and maintain
knowledge in such a scripting language, and requires implementing error
checking and possibly error recovery in the scripting language. Sure.. it
isn't strictly necessary, but I would rather just call a script
"restart_system.bash" that makes a call to a single ansible playbook
instead of seeing the bash script make a call to 6 ansible playbooks. What
happens if the playbooks need to pass state or other data between each
other? Much cleaner if it occurs in a single call, no?

That doesn't work because all plays in the playbook are filtered by the
tags provided, and each play needs to have a particular tag applied (first
stop *then* start later), not all of them.

My conclusion? *As of 1.8.4, roles are inadequate for complex playbooks
and include files need to be used instead.*

Roles are pretty simple--for more complex purposes, you may want to mix

in playbook includes too. That's fine! I wouldn't get too hung up on
getting all of the behavior for a certain part of your app in a single
role. That's going to be confusing and complicated to use.

That's unfortunate for code reuse reasons, etc. I would love to hear that

this is not the case and I have overlooked something. Maybe a feature to
resolve this is in the release plan?

I can't figure out how to do this even after reading the documentation
repeatedly, reading the ansible O'Reilly book (pre-release copy), the code
in ansible-examples, and various other resources.

I'm not sure if I understand any specific requests for improvement

here--besides being able to use a single playbook to drive multiple
operations, which is not really Ansible-style. If I've misunderstood, feel
free to let me know! :slight_smile:

One request would be that more complex cases/uses of Ansible be documented

more clearly, and specifically what you wrote above regarding the use of
"sub-roles": web/setup, web/restart, etc.

-Tim

--
You received this message because you are subscribed to the Google Groups
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to ansible-project+unsubscribe@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/ansible-project/CAH4wdVWc73Hc9x19AmiZ2CbZNJZPW2FyKv%3D7g-w88qnQtQNeQQ%40mail.gmail.com
<https://groups.google.com/d/msgid/ansible-project/CAH4wdVWc73Hc9x19AmiZ2CbZNJZPW2FyKv%3D7g-w88qnQtQNeQQ%40mail.gmail.com?utm_medium=email&utm_source=footer&gt;
.

For more options, visit https://groups.google.com/d/optout.

I wrote some test roles to verify what you described would work:

ansible$ find roles/{web,app1,app2} -type f | sort
roles/app1/startServer/tasks/main.yml
roles/app1/stopServer/tasks/main.yml
roles/app2/startServer/tasks/main.yml
roles/app2/stopServer/tasks/main.yml
roles/web/startMaintenance/tasks/main.yml
roles/web/startServer/tasks/main.yml
roles/web/stopMaintenance/tasks/main.yml
roles/web/stopServer/tasks/main.yml

All of those files have what amounts to the the following code:

ansible$ cat roles/web/startMaintenance/tasks/main.yml
---
- name: Run first task for startMaintenance
  debug: msg="Doing first task"
- name: Run second task for startMaintenance
  debug: msg="Doing second task"

Each role gets a playbook for testing and when separate execution happens
to be needed:

ansible$ find playbooks/test/ -type f | grep -v restart | sort
playbooks/test/app1_startServer.yml
playbooks/test/app1_stopServer.yml
playbooks/test/app2_startServer.yml
playbooks/test/app2_stopServer.yml
playbooks/test/web_startMaintenance.yml
playbooks/test/web_stopMaintenance.yml

Each of those has some trivial, essentially identical code:

ansible$ cat playbooks/test/web_startMaintenance.yml
---
- hosts: 127.0.0.1 # Imagine 'web'
  connection: local

  roles:
  - web/startMaintenance

Finally.. the restart playbook:

ansible$ find playbooks/test/ -type f | grep restart
playbooks/test/restart_system.yml
ansible$ cat playbooks/test/restart_system.yml
---
- include: web_startMaintenance.yml
- include: app1_stopServer.yml
- include: app2_stopServer.yml
- include: app2_startServer.yml
- include: app1_startServer.yml
- include: web_stopMaintenance.yml

Is there something about the way restart_system.yml is written above that
makes it "not Ansible-style" or that should be expected to lead to
problems? Thanks again.

4

Hey Jack,

5

I also find that smaller, job-oriented roles work well in scenarios like
this. It makes playbooks more maintainable and they read better when you
come back to them 3 months later.

6

you can have it both ways, you can have smaller playbooks that do part
of those functions and a wrapper one that includes the smaller ones,
this is a common pattern with site.yml (run everything).

7

Hey Jack,

The entire mission of Ansible is “simple IT automation”, right? :slight_smile: Wherever possible, my preference would be for a single Ansible playbook to handle the execution of a single high-level “goal”/“user request”. If that isn’t the case, then one needs to somehow bundle those externally, typically in a script. That requires the local “Ansible team” to decide on and maintain knowledge in such a scripting language, and requires implementing error checking and possibly error recovery in the scripting language. Sure… it isn’t strictly necessary, but I would rather just call a script “restart_system.bash” that makes a call to a single ansible playbook instead of seeing the bash script make a call to 6 ansible playbooks. What happens if the playbooks need to pass state or other data between each other? Much cleaner if it occurs in a single call, no?

Sorry, I think I misunderstood your intent here. I definitely agree with a single playbook for a single high level user request. I jumped to the conclusion that you wanted to encode a bunch of different user requests in a single playbook.

I wrote some test roles to verify what you described would work:

ansible$ find roles/{web,app1,app2} -type f | sort
roles/app1/startServer/tasks/main.yml
roles/app1/stopServer/tasks/main.yml
roles/app2/startServer/tasks/main.yml
roles/app2/stopServer/tasks/main.yml
roles/web/startMaintenance/tasks/main.yml
roles/web/startServer/tasks/main.yml
roles/web/stopMaintenance/tasks/main.yml
roles/web/stopServer/tasks/main.yml

All of those files have what amounts to the the following code:

ansible$ cat roles/web/startMaintenance/tasks/main.yml

  • name: Run first task for startMaintenance
    debug: msg=“Doing first task”
  • name: Run second task for startMaintenance
    debug: msg=“Doing second task”

Each role gets a playbook for testing and when separate execution happens to be needed:

ansible$ find playbooks/test/ -type f | grep -v restart | sort
playbooks/test/app1_startServer.yml
playbooks/test/app1_stopServer.yml
playbooks/test/app2_startServer.yml
playbooks/test/app2_stopServer.yml
playbooks/test/web_startMaintenance.yml
playbooks/test/web_stopMaintenance.yml

Each of those has some trivial, essentially identical code:

ansible$ cat playbooks/test/web_startMaintenance.yml

  • hosts: 127.0.0.1 # Imagine ‘web’
    connection: local

roles:

  • web/startMaintenance

Finally… the restart playbook:

ansible$ find playbooks/test/ -type f | grep restart
playbooks/test/restart_system.yml
ansible$ cat playbooks/test/restart_system.yml

  • include: web_startMaintenance.yml
  • include: app1_stopServer.yml
  • include: app2_stopServer.yml
  • include: app2_startServer.yml
  • include: app1_startServer.yml
  • include: web_stopMaintenance.yml

Is there something about the way restart_system.yml is written above that makes it “not Ansible-style” or that should be expected to lead to problems? Thanks again.

I would probably structure this a little bit differently, going back a little bit on what I said above about a single playbook per user request. This might actually be a really good use case for tags.

playbooks/test/restart_system.yml:

  • hosts: 127.0.0.1
    connection: local

roles:

  • { role: web/startMaintenance, tags: [‘web’] }

  • { role: app1/stopServer, tags: [‘app1’] }

  • { role: app2/stopServer, tags: [‘app2’] }

  • { role: app1/startServer, tags: [‘app1’] }

  • { role: app2/startServer, tags: [‘app2’] }

  • { role: web/stopMaintenance, tags: [‘web’] }

This is a fine refactor of my demo code, but the real production code can’t do this because app1 is on “app1 server cluster” and app2 is on “app2 server cluster”, etc You need a separate play for each call so that you can use the “hosts” field to indicate which server set it is being applied to. Right?

Now you can invoke it with:

ansible-playbook restart_system.yml --tags app1

Thanks for pointing this out.

…to only restart app1, or without tags to restart everything. (Maybe the web start/stop should be untagged, so it runs every time–not sure. Depends on your app.)

This way you lose the intermediary playbooks, and end up with a single playbook to do all of your restart tasks.

I don’t dislike the intermediary playbooks, since they can be used by themselves and more easily composed together.

If app1 and app2 roles are really close together, maybe you could parameterize them, too:

  • { role: genericApp/stopServer, appName: “app1”, tags: [“app1”] }

I will try to find an appropriate place to use this technique. Thanks!