Plugins in roles

Archives Ansible Developer
1

Sorry if that's been already been discussed...

It's currently possible to include modules under <rolename>/library and,
surprisingly, it's not possible to do the same with plugins, i.e. to put
them into the respective directories:

  <rolename>/action_plugins
  <rolename>/lookup_plugins
  <rolename>/callback_plugins
  <rolename>/connection_plugins
  <rolename>/filter_plugins
  <rolename>/vars_plugins

I'm ready to send a pull request that enables this behaviour.

Rationale:

1. Roles sometimes require additional Jinja filters, etc.

2. Manual installation of the plugins to the respective subdirectories
relative to the playbook or even system-wide plugin directories is
cumbersome. It causes a lot of management burden and it defeats the
grand idea of easily distributable and reusable roles. Ansible Galaxy
and automatic role installation is broken.

3. One can argue that we could just submit new plugins to the core but
those:
   - might not be admitted fast enough - devs have a lot of work
   - should not be ever submitted - as they make little or no
     sense for any other role and will simply clutter up the core

4. And finally <rolename>/library is already allowed, possibly due to
the similar reasons.

What are your opinions?

2

Hi Jakub,

Generally speaking Ansible wants to encourage core contribution. Even though we can include custom code in a “./library” directory, I don’t want that to reduce someone’s interest in adding something to core, where everyone can find it and maintain it, rather than creating a lot of tiny unsupported (or lightly supported) modules. This kind of thing can just add to frustration with roles and is one of the things I’ve seen in working with similar community hubs.

That being said, it’s something we support, and there are uses for it.

With roles and other types of plugins, I again worry about the same sort of thing, and in general, I don’t want to build a system that requires lots of additions to Ansible in terms of custom filters and loop types and so on. If we are doing that, I feel we are doing it a bit wrong. That being said, it’s also something we support.

However, these plugins are really global, so I think the right way here is to still install them in their configured paths, and make it more explicit that you are adding these things.

I definitely would not like to see a Galaxy plugin add a new connection type and someone suddenly find out ansible is doing something different because someone would like it to be “different”, etc.

Perhaps this could be something where Galaxy has lots of warnings on a roles page when it brings along extra modules or plugins, and that might be a way to address it.

In any event, you probably shouldn’t find the omission too suprising – Ansible tries to follow YAGNI and MVP in most things, so we’ll add things later after we start to see a lot of requests for them.

Right now, this is not something I see a lot of requests for, and I worry a bit about what that encourages. (for instance, if something isn’t in 1.5, does someone include it in 1.4 as a one-off, and then when 1.5 comes out, the role is forever using the old unmaintained code that’s part of the role?)

I think in many cases if you have to write a lot of code to extend Ansible in interesting ways, I would like to have that discussion here about what’s missing in core, rather than shunting things immediately to the plugin system and not being able to have that conversation.

In your case, what were you wishing to include in a role, etc?

3

In your case, what were you wishing to include in a role, etc?

Consider the following example of a Jinja filter:

def to_ipv6(ipv4):
    """Convert IPv4 to IPv6.
    
    >>> to_ipv6('192.0.2.3')
    '2002:c000:0203::1'
    
    """
    parts = ipv4.split('.')
    intparts = map(int, parts)
    fmt = "2002:{:02x}{:02x}:{:02x}{:02x}::1"
    ipv6 = fmt.format(*intparts)
    return ipv6

1. I use it a couple of times in a few templates of my role.
2. I don't see any way to achieve it with a generic Jinja/Ansible
features. Even if it was, it would be a nightmare (DYI!, readability) to
build a complex pipeline of filters and copy&paste everywhere.

To be honest, it doesn't make much sense if you allow to include plugins
in the playbook directory, and not to allow it in the role... Also, what
about situations when one has a filter that they do not want to share
with the public, but still be able to easily reuse, in, say, closed
corporation environment?

4

You can still put them in the plugins dir.

5

The role is meant to be reusable and easily installable through Galaxy.
There is no way to distribute plugins with roles. So how would you
achieve it?

6

​I think Jakub has a point here.
​While i agree it's better to have modules pushed to core, there's the
reality that not all of them will be.

I very much like this to be considered.

  Serge

7

I don’t know where one might get the vibe that it hasn’t been considered given my very long response above :slight_smile:

Ultimately, it’s also not the most important thing we need to do right now.

Ansible does have a batteries included philosophy for a reason – to force the discussion about expanding core versus causing divergence. If many roles wanted something similar, how many copies would be in circulation, etc? Those are some of the factors we are weighing.

In the case of the ipv6 filter, I’m not sure why this wouldn’t even be fine for core ansible – it’s pretty small. I can’t remember if I made a call on that earlier or not, but it doesn’t seem problematic.

Ultimately we’ve simply got too much going on right now to be adding too many new things just yet – I think right now the most important thing for the project – for people who do want to get more new things in – would be for all the folks having ideas to focus on closing bugs – and testing pull requests – and then we’ll get back to considering additions. Take a look at github and you’ll see what I mean :slight_smile:

Currently the number of bugfix PRs outnumbers pull requests for features by 7x and we still have to get through all of those first.

And it’s definitely not in a state where we can just merge all of these, as many of these need revisions.

Patience is greatly appreciated!

8

Ok, old post. But I just now am running into this.

So there is no way for a role to define and use a plugin as parts of its implementation? If so, how does one reuse plugins that are not in core?

And if a role did require say a filter plugin (that it was not allowed to implement itself) how would role work? Would there need to be documentation in the role telling the role user to please copy some plugin code in the playbook plugins directory?

I also did not quite follow the argument as to why having plugins in a role is a bad idea? Because it would discourage contributions to core and cause duplication? If so, then that same argument could be used against roles themselves.

9

I'm not sure what you've been reading, but this has been working for a
long time.

Roles can include all kinds of plugins, they just need to be in the
correctly named directory ('library' for task plugins,
'filter_plugins' for filter plugins, etc). Once the role is loaded the
plugins are available for the full play. This also works with
directories adjacent to plays or you can even change the task module
directory paths in ansible.cfg.

10

Hmm, ok. I tried placing my plugin in role/filter_plugins but it was not found. So I started searching. Could not find any docs on roles+plugins but did find this thread.

Follow Q then: Can a filter_plugin get access to facts/variables?

11

sorry, this was the case, it seems to be currently broken. The feature
is not widely used and mostly for 'task plugins' which still work.

Working on fixing and re-enabling this again.

12

Brian,

I was wondering what the status of this fix is?
I am looking at using a role lookup plugin and adding it to role/lookup_plugins doesn’t work. Will this be included in the 1.9.2 release?

Thank you kindly,
Rumen Telbizov

13

It will not, but it will be included in 2.0, and is currently operational in the devel branch.

14

Great news, waiting for 2.0 to get this feature.

15

Hi All,

is role/action_plugins supported in recent ansible versions?

thank you,
Daniel

16

As of v2, this should be supported.