# generic 'package' module? **URL:** \ **Category:** Ansible Project\ **Created:** [June 7, 2015, 3:05pm UTC](https://forum.ansible.com/t/generic-package-module/18425 "2015-06-07T15:05:05Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Michael\_Johnson1](https://avatars.discourse-cdn.com/v4/letter/m/dec6dc/32.png) [@Michael\_Johnson1](https://forum.ansible.com/u/Michael_Johnson1)\ **Post date:** [June 7, 2015, 3:05pm UTC](https://forum.ansible.com/t/generic-package-module/18425/1 "2015-06-07T15:05:05Z")
It has always sort of bothered me that ansible does not have a generic ‘package’ module that you then can tell it what ‘provider’ to use if you want to override the default like other tools such as puppet. While doing some searching on the topic, I found this: [https://github.com/ansible/ansible/issues/4261](https://github.com/ansible/ansible/issues/4261) That issue asks about this same question and the core of the response seems to be the last part of the last sentence: “as you have to make exemptions so many times, it really causes problems down the road.” While I can somewhat understand this philosophy, it does not seem to been one that is internally consistent. Take the ‘service’ module for example. Services can require just as many exemptions as packages. If this philosophy is really the sticking point, why don’t we have ‘systemd\_service’, ‘smf\_service’, ‘upstart\_service’, etc.? There may very well be a good explanation for the apparent inconsistency, but I am not seeing it. Can anyone explain why these two things which seem very similar to me are treated very differently in ansible? Would the ansible project be open to having a unified ‘package’ module? ---
**Author:** ![sivel](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sivel/32/1198_2.png) [@sivel](https://forum.ansible.com/u/sivel)\ **Post date:** [June 7, 2015, 7:02pm UTC](https://forum.ansible.com/t/generic-package-module/18425/2 "2015-06-07T19:02:58Z")
You can already do: - action: “{{ ansible\_pkg\_mgr }}” name=some-pkg-name ---
**Author:** ![Brian\_Coca](https://avatars.discourse-cdn.com/v4/letter/b/6de8d8/32.png) [@Brian\_Coca](https://forum.ansible.com/u/Brian_Coca)\ **Post date:** [June 7, 2015, 9:25pm UTC](https://forum.ansible.com/t/generic-package-module/18425/3 "2015-06-07T21:25:17Z")
Yes, it is a contradiction, sadly it is very hard to remove 'service' now, it is a huge pathway of detection and fallbacks and I would be very happy to create a systemd, upstart, initd and other modules to substitute it. You just have to look at the tickets to see that every time we do a minor modification to service it has many unintended consequences, it is fragile, hard to read and very hard to debug. That and other reasons made me keep I kept the svc/daemontools module separate. ---
**Author:** ![Michael\_Johnson1](https://avatars.discourse-cdn.com/v4/letter/m/dec6dc/32.png) [@Michael\_Johnson1](https://forum.ansible.com/u/Michael_Johnson1)\ **Post date:** [June 7, 2015, 10:52pm UTC](https://forum.ansible.com/t/generic-package-module/18425/4 "2015-06-07T22:52:00Z")
Ah! That works wonderfully! ---
**Author:** ![Michael\_Johnson1](https://avatars.discourse-cdn.com/v4/letter/m/dec6dc/32.png) [@Michael\_Johnson1](https://forum.ansible.com/u/Michael_Johnson1)\ **Post date:** [June 7, 2015, 11:00pm UTC](https://forum.ansible.com/t/generic-package-module/18425/5 "2015-06-07T23:00:44Z")
Got it, this makes perfect sense. Always nice to have the background on inconsistencies like this (for me a least it removed a mental block). With this background coupled with the notion of using ‘action’, perhaps ‘service’ could be deprecated and eventually removed in favor of different modules. ---
**Author:** ![Dick\_Davies](https://avatars.discourse-cdn.com/v4/letter/d/b9e5f3/32.png) [@Dick\_Davies](https://forum.ansible.com/u/Dick_Davies)\ **Post date:** [June 8, 2015, 7:18am UTC](https://forum.ansible.com/t/generic-package-module/18425/6 "2015-06-08T07:18:57Z")
Good point on the disparity (and despite generally working pretty well for a big case statement, service will occasionally give confusing errors when it guesses wrong). Would it make sense to migrate to splitting out each service subtype into its own module? I might be being a bit optimistic but it'd allow a phased deprecation - there are no existing modules called 'upstart', 'init' etc so that should be straightforward (right?). ---
**Author:** ![Brian\_Coca](https://avatars.discourse-cdn.com/v4/letter/b/6de8d8/32.png) [@Brian\_Coca](https://forum.ansible.com/u/Brian_Coca)\ **Post date:** [June 8, 2015, 2:43pm UTC](https://forum.ansible.com/t/generic-package-module/18425/7 "2015-06-08T14:43:52Z")
Yep, this is what I am thinking .. just need the time and resources to make it so. ---
**Author:** ![Adam\_Kosmin](https://avatars.discourse-cdn.com/v4/letter/a/dc4da7/32.png) [@Adam\_Kosmin](https://forum.ansible.com/u/Adam_Kosmin)\ **Post date:** [June 8, 2015, 5:02pm UTC](https://forum.ansible.com/t/generic-package-module/18425/8 "2015-06-08T17:02:16Z")
I have my doubts as to how wise this approach is given the different set of parameters supported by each package module. I feel much better stuffing these OS-specific modules into their own files which get included based on things such as ansible\_os\_family. Best, Adam ---
**Author:** ![Brian\_Coca](https://avatars.discourse-cdn.com/v4/letter/b/6de8d8/32.png) [@Brian\_Coca](https://forum.ansible.com/u/Brian_Coca)\ **Post date:** [June 8, 2015, 5:15pm UTC](https://forum.ansible.com/t/generic-package-module/18425/9 "2015-06-08T17:15:55Z")
@Adam, Agreed, this workaround just offsets the work of having 2 tasks to having 2 data structures and is more limited than the other version, this is one of the reasons why we do not recommend it, but it is available if people insist going down that path. ---
**Author:** ![Christian\_Thiemann](https://avatars.discourse-cdn.com/v4/letter/c/f04885/32.png) [@Christian\_Thiemann](https://forum.ansible.com/u/Christian_Thiemann)\ **Post date:** [June 10, 2015, 3:43pm UTC](https://forum.ansible.com/t/generic-package-module/18425/10 "2015-06-10T15:43:44Z")
I want all my Apache installation to behave the same on all OSs – I want to write one logical model of how Apache should work, and all OS differences should just be different views of that model. Thus, this is how my “install Apache packages” task looks like – one task, one data structure, thanks to some YAML magic: ` - include: install\_packages.yml pkgs: !platform - apache:: Debian: apache2 RedHat: httpd - mod\_authnz\_external:: Debian: libapache2-mod-authnz-external - python-passlib:: # needed for Ansible’s htpasswd module Gentoo: dev-python/passlib - pwauth tags: - apache/setup ` If I had separate include files per OS, then my list of packages would be duplicated in multiple files – I mean the logical list of packages (i.e., “apache” + “mod\_authnz\_external” + “python’s passlib”). If I have to add another mod\_foobar in the future, I only have to add it to one list instead of having to remember to maintain multiple OS-dependent representations in multiple places. ---
**Author:** ![Giovanni\_Tirloni](https://avatars.discourse-cdn.com/v4/letter/g/71c47a/32.png) [@Giovanni\_Tirloni](https://forum.ansible.com/u/Giovanni_Tirloni)\ **Post date:** [June 10, 2015, 4:13pm UTC](https://forum.ansible.com/t/generic-package-module/18425/11 "2015-06-10T16:13:07Z")
All this magic feels like more typing and wasted brain cells than keeping separate lists per distribution. I've come to the conclusion that in Ansible you either hide complexity in modules, roles or playbooks, in that order. If the module isn't featured/mature enough (and you don't want or can't improve it), you write roles that do the magic you need. It gets all hidden from you and the playbooks look very clean. Otherwise you let the imperfection fall through and deal with it in the playbooks, with command-line options and what not. I don't know what's best for your situation. It seems a role encapsulating these "imperfections" might be enough.. or you could write a module for completely managing Apache (from install to deploying new websites). They say, Ansible is a radically simple IT automation engine... I think it's much more flexible than simple (perhaps simpler compared to the absolute insanity of other automation tools). That comes with its drawbacks. Giovanni ---
**Author:** ![Dick\_Davies](https://avatars.discourse-cdn.com/v4/letter/d/b9e5f3/32.png) [@Dick\_Davies](https://forum.ansible.com/u/Dick_Davies)\ **Post date:** [June 10, 2015, 4:34pm UTC](https://forum.ansible.com/t/generic-package-module/18425/12 "2015-06-10T16:34:30Z")
I've seen that attempted with Puppet and it works out badly in practice. Different distros put files in different locations, use different init systems, have different conventions for 'includes', name packages differently. Having one 'magic' class that makes it easy to type a config seems like it'll save you time. until you have to debug it. ---
**Author:** ![Brian\_Coca](https://avatars.discourse-cdn.com/v4/letter/b/6de8d8/32.png) [@Brian\_Coca](https://forum.ansible.com/u/Brian_Coca)\ **Post date:** [June 10, 2015, 5:13pm UTC](https://forum.ansible.com/t/generic-package-module/18425/13 "2015-06-10T17:13:15Z")
So to put this to bed: [http://docs.ansible.com/package\_module.html](http://docs.ansible.com/package_module.html) I still think it is not what you want or need, but now you have the option. ---
**Author:** ![Adam\_Kosmin](https://avatars.discourse-cdn.com/v4/letter/a/dc4da7/32.png) [@Adam\_Kosmin](https://forum.ansible.com/u/Adam_Kosmin)\ **Post date:** [June 10, 2015, 8:30pm UTC](https://forum.ansible.com/t/generic-package-module/18425/14 "2015-06-10T20:30:57Z")
Brian, Very cool. I am personally a big fan of this effort and am curious to see how it evolves. Dick, C’mon man, that’s not true at all. The “works out poorly in practice” portion of your comment is more about any given implementation rather than the tool itself. We all know any of these tools _can_ get the job done and in most cases, the cobweb of damage is constructed by the persons using it. I myself have used this type of abstraction to manage thousands of distributed systems with several OS’s in site-specific areas all over the country. There are many other people out there with similar successes and I’d argue each of those wins goes back to 3 things: tools, design, and work flow. No tool is going to be a magic cure-all when proper design and work flows are lacking. Best, Adam ---
**Author:** ![Brian\_Coca](https://avatars.discourse-cdn.com/v4/letter/b/6de8d8/32.png) [@Brian\_Coca](https://forum.ansible.com/u/Brian_Coca)\ **Post date:** [June 10, 2015, 9:07pm UTC](https://forum.ansible.com/t/generic-package-module/18425/15 "2015-06-10T21:07:48Z")
The reason this works poorly in practice is that the undertaking is like trying to set in stone a moving target. packages are not named the same, the same app can appear under various names packages are not split the same way, one app can be 1 package in one distro or 3 in another packages don't stay the same inside of a distro, they get renamed, split, joined, etc, this mostly happens at distro version boundaries, but not always distros do not always have version boundaries, rolling release distros! packages with the same name are not always the same app even more fun, throw in virtual and meta packages, package groups! some distros name per language milestone python2.5-package python3.4-package others cut off at major versions py2-, py3, others at old/current py2, py- now add external/custom repos outside the tighter control of the distributions and hope that when they overlap ... it is actually what you want. now add per programming language package managers, some languages have multiple! now try to remember what my sanity looked like. This is what people really want solved, they want to say 'install apache' and have something magically navigate the above quagmire and give them what they wanted, not what they asked for. Sadly that is much more than what an ansible module was designed to do, you required a central database that is constantly updated and amended with every variation, OS, distribution and version, a 'wikipackagesmedia' which is a much larger project than I'm willing to attempt at this point in time. ---
**Author:** ![Christian\_Thiemann](https://avatars.discourse-cdn.com/v4/letter/c/f04885/32.png) [@Christian\_Thiemann](https://forum.ansible.com/u/Christian_Thiemann)\ **Post date:** [June 10, 2015, 9:09pm UTC](https://forum.ansible.com/t/generic-package-module/18425/16 "2015-06-10T21:09:55Z")
I don’t necessarily want to save time, but have a well-designed setup that saves me from having headaches / making mistakes later on. Take the following task from my apache role: ` - name: “disable SSLv2 & SSLv3” lineinfile: dest: !platform Debian: /etc/apache2/mods-available/ssl.conf Gentoo: /etc/apache2/vhosts.d/00\_default\_ssl\_vhost.conf RedHat: /etc/httpd/conf.d/ssl.conf regexp: “{{ item.regexp }}” line: “{{ item.line }}” with\_items: - { regexp: '^\s\*SSLProtocol ', line: ‘SSLProtocol ALL -SSLv2 -SSLv3’ } notify: - reload apache tags: - apache/setup ` It’s our policy to not use SSLv2 or v3. That policy is expressed in one place (and one place only). If I had one lineinfile task for each OS (possibly in separate files), I’d have to duplicate the policy to all those places. If we want to disable TLS 1.0 in the future, it would have to be changed in multiple places – something that programmers avoid rigorously, so why should it be different in a devops context? The thing I want to achieve is “install Apache on a server”, not “install Apache on Debian” and “install Apache on RedHat” and “install Apache on Gentoo”. Ansible (and other cfg mgmt systems as well, I assume) imposes quite some annoying restrictions on ops work (“it would be so much easier to just SSH in and fix this manually”), but that’s ok because it provides reproducibility and system documentation – and OS abstraction. To me, that is one of the major points of using config management in the first place – reproducibility and documentation can be achieved using shell scripts, possibly with less effort than by using cfg mgmt. I’d say that the OS abstraction in ops is like the database abstraction layer in dev work. I’ve seen a few posts in this group which recommend creating separate Apache roles for each distro – to me, they all sound like: “Database abstraction is really hard – if your application needs to support both MySQL and Postgres as a backend, you should create two totally independent forks of your application, one for each database.” @Brian: I think the package module is a very good thing. For most use cases, this is the perfect module – for everything else one can still use yum, apt, etc. in combination with “when: ansible\_os\_family == ‘foobar’”. The same design could be applied to the service module: split into separate modules so you can use all the fancy features of specific init systems if you want/need to, but also provide an “ansible\_svc\_mgr” fact and a “service” module / action plugin that work like “ansible\_pkg\_mgr” and the new “package” module. ---
**Author:** ![Brian\_Coca](https://avatars.discourse-cdn.com/v4/letter/b/6de8d8/32.png) [@Brian\_Coca](https://forum.ansible.com/u/Brian_Coca)\ **Post date:** [June 10, 2015, 9:17pm UTC](https://forum.ansible.com/t/generic-package-module/18425/17 "2015-06-10T21:17:21Z")
@Christian, those are my thoughts for the future of the service module, this package module will serve as a template on how to split it and keep backwards compatibility using the generic module as a placeholder. Just need to add service system detection to facts and then use those internally. As for keeping the policy in a single place, the same can be achieved by just making the ssl cipher string into a common variable and then have each template reference it. This does not work for all cases but I find it better to keep specific tasks for each OS and then abstract the common data, than the reverse which abstracts the common task but requires abstracting the non-common data. Again, I'm all for choice, in this case I think it is an illusion as you update 1 common yaml file and 2 non-common yaml files vs 2 common yaml files and 1 non-common yaml file. ---
**Author:** ![Christian\_Thiemann](https://avatars.discourse-cdn.com/v4/letter/c/f04885/32.png) [@Christian\_Thiemann](https://forum.ansible.com/u/Christian_Thiemann)\ **Post date:** [June 10, 2015, 9:57pm UTC](https://forum.ansible.com/t/generic-package-module/18425/18 "2015-06-10T21:57:51Z")
> @Christian, those are my thoughts for the future of the service > module, this package module will serve as a template on how to split > it and keep backwards compatibility using the generic module as a > placeholder. Just need to add service system detection to facts and > then use those internally. Cool! > As for keeping the policy in a single place, the same can be achieved > by just making the ssl cipher string into a common variable and then > have each template reference it. This does not work for all cases but > I find it better to keep specific tasks for each OS and then abstract > the common data, than the reverse which abstracts the common task but > requires abstracting the non-common data. That’s only a partial solution – you still have multiple copies of the policy “we set the SSL cipher string to something custom.” The thing I want to achieve is “disable SSLv3 on our Apache servers”, not “define a cipher string that disables SSLv3” and “set that cipher string on Debian” and “set that cipher string on RedHat” etc. Say, you want to add support for Arch in the future. You add another include file but happen to forget to add the lineinfile task. Your playbooks run fine, POODLE happened a long time ago and you don’t even think about SSLv3 anymore, but all your Arch servers will offer it (assuming Arch’s default config still has it enabled, of course…). My playbooks will fail on Arch hosts if I forget to add Arch’s config filename to my “disable SSLv3” task, reminding me about my “no SSLv3” policy. > Again, I’m all for choice, in this case I think it is an illusion as > you update 1 common yaml file and 2 non-common yaml files vs 2 common > yaml files and 1 non-common yaml file. I am only editing 1 common yaml file – roles/apache/tasks/main.yml – that includes all the relevant data (“!platform” is a YAML tag that I add to the YAML parser (mis)using a vars\_plugin, which, granted, is somewhat hacky and will hopefully still work in v2 😉 ). Re your other comment about package versions: yes, we will always have to manually curate some OS details. But cfg mgmt should help making that easier, not say “package NAMES are different anyways, so we won’t even let you abstract the package MANAGER.” My task for installing PHP (different number of packages in different distros) looks like this: ` - include: install\_packages.yml pkgs: !platform - php:: Debian: php5 RedHat: php55w - Debian: php5-cli RedHat: php55w-cli - Debian: php5-curl notify: - reload apache tags: - apache/php/setup ` That installs the logical “php” package (which is provided by “php5” on Debian, and “php55w” on RedHat, and “php” everywhere else, i.e. Gentoo in my case). In addition, it will install the CLI SAPI on Debian and RedHat (that’s included in “php” on Gentoo), and the curl extension on Debian (included in “php55w” on RedHat). All in one single yaml file. Different versions of distros can be treated as distinct distros. Until recently (before we killed our last Ubuntu 10 and 12 servers), I had this in my Apache role: ` lineinfile: dest: !platform Ubuntu14: /etc/apache2/sites-enabled/000-default.conf Ubuntu12: /etc/apache2/sites-enabled/000-default Ubuntu10: /etc/apache2/sites-enabled/000-default ` This wasn’t perfect: there’s still some duplication, but at least it’s very close together and not spread over multiple files. Rolling-release distributions are tricky – luckily I am able to keep all my (few) Gentoo servers in sync, so I don’t need to support multiple “versions” of Gentoo. ---
**Author:** ![Brian\_Coca](https://avatars.discourse-cdn.com/v4/letter/b/6de8d8/32.png) [@Brian\_Coca](https://forum.ansible.com/u/Brian_Coca)\ **Post date:** [June 10, 2015, 10:05pm UTC](https://forum.ansible.com/t/generic-package-module/18425/19 "2015-06-10T22:05:50Z")
Any intent on sharing that vars plugin? it sounds like a nice way to unify vars in a single by distro vs per distro file. ---
**Author:** ![Christian\_Thiemann](https://avatars.discourse-cdn.com/v4/letter/c/f04885/32.png) [@Christian\_Thiemann](https://forum.ansible.com/u/Christian_Thiemann)\ **Post date:** [June 10, 2015, 10:39pm UTC](https://forum.ansible.com/t/generic-package-module/18425/20 "2015-06-10T22:39:57Z")
Yes, been meaning to do that for some time, but haven’t gotten around to properly cleaning it up etc. Happy to create a PR if you are interested – for now, here’s the rough version: > The filter\_plugin returns the value of a dictionary that exists for a list of keys. Example: ` # vars file foo: Debian: bar Ubuntu: baz Gentoo: abc # Jinja template {{ foo|selectby([‘Gentoo’]) }} == abc {{ foo|selectby([‘Debian’, ‘Ubuntu’]) }} == bar {{ foo|selectby([‘Ubuntu’, ‘Debian’]) }} == baz ` The vars\_plugin defines a variable for each host that’s a list of ansible\_distribution + version, ansible\_os\_family, etc. in descending order of specificity, and modifies the YAML parser to recognize dictionaries tagged !platform, which will be converted into Jinja templates using the |selectby filter. Example: ` # vars file foo: !platform Debian: bar Ubuntu: baz Gentoo: abc # Jinja template {{ foo }} == abc if ansible\_os\_family == ‘Gentoo’ {{ foo }} == bar if ansible\_os\_family == ‘Debian’ and not ansible\_distribution == ‘Ubuntu’ {{ foo }} == baz if ansible\_distribution == ‘Ubuntu’ ` The |selectby filter is a bit more complex than just this, it also transparently iterates over lists, so that these two are equivalent: ` foo1: - !platform Debian: bar - !platform Debian: barbara foo2: !platform - Debian: bar - Debian: barbara ` And it allows to define a default value: ` foo: !platform default-value:: Debian: override-value {{ foo }} == (‘override-value’ if ansible\_os\_family == ‘Debian’ else ‘default-value’) ` [Next page](https://forum.ansible.com/t/generic-package-module/18425.md?page=2)