Does anyone else think it would be nice for the yum and apt modules to take a list of packages, e.g.:
apt pkg=vim=1.0,nano=2.0 state=installed
I know there might be some objections to doing more than one thing per action, but it is also often desirable to install a whole list of packages all at once.
This has been discussed some, and discussed more on IRC lately. I
think the answer to this is yes -- people want this. I have been
resisting mainly due to syntax reasons, which I think I have an answer
for.
with_items is a very nice feature, but due to the way the package
managers work, it is too much calculation to be particularly zippy.
Though I still want to be able to write lists that way, a huge comma
seperated list of packages kind of sucks.
Make it so. If one module changes, both will need to. Change
handlers and everything must continue to work.
Syntax wise, I'd really like to have something like this:
- action: foo pkg=$items ...
with_items:
- nano=2.0
- vim=1.0
What's new here? This is a bit sketchy to implement, but I think if
with_items sees that the variable being used in the action line is
"$items" and not "$item", it could join all the values together versus
turning them into multiple tasks. The result of this would be it
would be TRIVIAL to pass in the list of packages a list or external
data structure.
So this should work as well:
- action: foo pkg=$items
with_items: $items
Thoughts?
I think this could be very nice in terms of "doing what you mean"
without introducing very confusing ways of having to turn lists into
comma seperated values with Jinja2.
I am increasingly believing if you have to see Jinja2 syntax outside
of templates, we've failed a bit... and we were talking today about
making the $variable operator do something for nested values that
would make it better for those kind of cases, like
${foo.ansible_eth0.foo}
That's a bit of a tangent but things start to dovetail together.
Let's figure out what the ideas are and I can open up some tickets on
the different aspects.
The way I’ve found around the with_items yum/apt problem is to make a dummy RPM that pulls in all the packages you’d want to iterate through with $item, and then configure yum on the target instance with a private repo. Then, use action: yum to pull in the dummy package.
Using yum for the heavy lifing because:
It would make it even simpler/faster/more beautiful if there were a low-level ‘package’ action that let me do the equivalent of ‘yum localinstall http://overlord.local:some_port/RPMS/quux.rpm?internal_and_private_ansible_secret=1234’ We could use Tornado on the overlord machine and serve ~ansible/packages/ off some_port, and then the playbook action could be as simple as ‘package rpm=quux’ (or ‘package deb=quux’ depending on the platform.) Then the package’s postinstall hookscript could do most of the heavy lifting. As a bonus, on Fedora you get 'yum history and ‘yum undo’. On both Fedora and Debian-derived systems, you also get package and repo keysigning. These are niceties that ansible lacks (and rightly so.)
@OP: Message me on G+ if you’d like an RPM .spec template suitable for use with ansible, but vim will happily make one for you if you open a file with the .spec suffix in the appropriate directory (~makerpm/rpmbuild/SPECS). I found this wiki page helpful for understanding the RPM format; I was an RPM specfile newb until a week or two ago, and now I’m happily rocking a private repo. Ansible generates the repo file with the ‘template’ action and notifies a handler that does ‘yum makecache’. Since I’ve also configured yum-updatesd via ansible, any future changes to the installation are one quick ‘rpmbuild’ away. I drop the new package in my private repo and it is pulled and updated within 3600 seconds. I’m going to make a script to generate an .rpm or a .deb from a YAML package list, and if Mike likes it maybe he’d accept the pull request. ;D
Speaking of which: although I think an ‘$items’ syntax would be great, one of the things about ansible I most appreciate is how the project (seems to) recognize how much of the work taken on by other DevOps CNC systems is wheel-reinvention. Apt-get appeared in what year? 1998 or something? We’ve had dependency resolution with event hooks since that long, folks. On Fedora, we’ve had it at least since the switch to Yum, possibly earlier (depending on your opinion of Up2date). We didn’t have to wait for Puppet and Chef. ;D Thanks, Mike, for winnowing the stack. Please don’t screw it up by adding a bunch of new stuff! ;D (I know you wouldn’t, that’s why I love ansible.)
~elsebeth
The way I’ve found around the with_items yum/apt problem is to make a dummy RPM that pulls in all the packages you’d want to iterate through with $item, and then configure yum on the target instance with a private repo. Then, use action: yum to pull in the dummy package.
for that issue – We really should just make the yum (and apt) module support taking a list. This has been discussed before and could be done with some tweaks in runner code such that with_items looks the same way it does now, but is evaluated differently.
Meta packages are a bit much to ask of people I think, but yes, that’s a decent workaround many people employ with various tools.
I’d say work it out with skvidal whether supporting a local install makes sense and whether pkg=/path/to/foo.rpm is the best way of expressing that. I don’t object but Seth gets to make the call as to whether he thinks it’s reasonable and messes with the other capabilities of the module too much.
Using yum for the heavy lifing because:
- you get dependency resolution and pre/post installation scripts for free
- it’s much, much, much faster than using ansible’s ‘yum’ action a zillion times
- you can sign your packages (creates accountability and raises security)
- you can ‘yum undo’ and ‘yum history’ to reverse adverse outcomes.
It would make it even simpler/faster/more beautiful if there were a low-level ‘package’ action that let me do the equivalent of ‘yum localinstall http://overlord.local:some_port/RPMS/quux.rpm?internal_and_private_ansible_secret=1234’ We could use Tornado on the overlord machine and serve ~ansible/packages/ off some_port, and then the playbook action could be as simple as ‘package rpm=quux’ (or ‘package deb=quux’ depending on the platform.) Then the package’s postinstall hookscript could do most of the heavy lifting. As a bonus, on Fedora you get 'yum history and ‘yum undo’. On both Fedora and Debian-derived systems, you also get package and repo keysigning. These are niceties that ansible lacks (and rightly so.)
Interesting. Trying to lock down licensed content or count installations, perhaps?
Last approach I saw to this in-house control was this: https://lists.dulug.duke.edu/pipermail/yum-devel/2006-November/002856.html
108 got slain. I’m really not sure how useful it would be either.
Speaking of which: although I think an ‘$items’ syntax would be great, one of the things about ansible I most appreciate is how the project (seems to) recognize how much of the work taken on by other DevOps CNC systems is wheel-reinvention. Apt-get appeared in what year? 1998 or something? We’ve had dependency resolution with event hooks since that long, folks. On Fedora, we’ve had it at least since the switch to Yum, possibly earlier (depending on your opinion of Up2date). We didn’t have to wait for Puppet and Chef. ;D Thanks, Mike, for winnowing the stack. Please don’t screw it up by adding a bunch of new stuff! ;D (I know you wouldn’t, that’s why I love ansible.)
I’m strongly not for adding a lot of new stuff, but I also don’t believe you should need to make a meta-package for all of your possible package combinations, or need to know rpmbuild to need to use Ansible efficiently.
For instance, if you want Cobbler + Nagios, a package that is “my-magic-managemnet-server.rpm” is a bit much. That’s not dependency resolution, that’s installing two different things on the same box.
So I think it’s still worthwhile to let those take lists, and to let “with_items” be magical for yum/apt…. if it just does what you mean, that’s ok.
But yeah, I will be pushing back on continual new syntax. What I like the most about this idea is it’s reusing syntax we already have.
Replies inline
Meta packages are a bit much to ask of people I think, but yes, that’s a decent workaround many people employ with various tools.
Yep. It would be cool if those tools were a lot easier to use, but that’s ‘just’ a user interface issue (famous last words)
For instance, I have a quick/dirty script ‘ansibile’ (note the extra i) that I call like this
ansibile @Development my-favourite-editor blahblah quux … > development.rpm
and bam, I get a package that installs all packages in the Development group, plus the rest of the named packages, and nothing else. Then later I make additions to the package list by pushing updates to development.rpm over yum-updatesd from my private repo. And I jumpstart that process with a simple ‘mv development.rpm ~repo/noarch’. It’s incredibly painless. I’m going to blog about my setup, it’s very handy and easy to use.
I’d say work it out with skvidal whether supporting a local install makes sense and whether pkg=/path/to/foo.rpm is the best way of expressing that. I don’t object but Seth gets to make the call as to whether he thinks it’s reasonable and messes with the other capabilities of the module too much.
Understood and appreciated, thanks!
Interesting. Trying to lock down licensed content or count installations, perhaps?
I do web apps and sites freelance, and I would get sued for breach-of-contract if I put people’s IP in a public repo. And they’d be right. 
I’m strongly not for adding a lot of new stuff, but I also don’t believe you should need to make a meta-package for all of your possible package combinations, or need to know rpmbuild to need to use Ansible efficiently.
True! But this is why we need a utility to generate them efficiently, and a yum-localinstall action to distribute them efficiently.
For instance, if you want Cobbler + Nagios, a package that is “my-magic-managemnet-server.rpm” is a bit much. That’s not dependency resolution, that’s installing two different things on the same box.
Exactly, yes, $items is a great step forward. Good thinking.
The way I've found around the with_items yum/apt problem is to make a
dummy RPM that pulls in all the packages you'd want to iterate
through with $item, and then configure yum on the target instance
with a private repo. Then, use action: yum to pull in the dummy
package.
I've never liked metapkgs b/c it makes tracking various things harder,
ultimately. However, they aren't the end of the world
Using yum for the heavy lifing because:
- you get dependency resolution and pre/post installation scripts
for free
- it's much, much, much faster than using ansible's 'yum' action a
zillion times
- you can sign your packages (creates accountability and raises
security)
- you can 'yum undo' and 'yum history' to reverse adverse outcomes.
you can use yum history for any given transaction, too.
It would make it even simpler/faster/more beautiful if there were a
low-level 'package' action that let me do the equivalent of 'yum
So two things here:
1. making yum handle multiple pkgs specified would be fine - my concern
is error reporting. I would want to specify the pkgs to the yum module
1 by 1 so I could accurately say 'pkg foo could not be installed' - How
should the failure report look for the yum module with multiple pkgs?
I'm happy to make it work - I just don't know what looks best here.
Also should a single pkg install failure stop the rest in that command
from running?
2. localinstall - I don't know what that would look like declaratively:
state=installed pkg=/some/pkg/over/here?
is that what you want?
The yum module is pretty simple, actually, - the only special thing it
is doing is to look up the pkg state before to make sure we can do what
we're being asked to do. That's for the idempotent requirement.
lemme know what you think here - I can make a number of things happen
in the module - just really need a 'spec' of what is wanted an expected.
-sv
1. making yum handle multiple pkgs specified would be fine - my concern
is error reporting. I would want to specify the pkgs to the yum module
1 by 1 so I could accurately say 'pkg foo could not be installed' - How
should the failure report look for the yum module with multiple pkgs?
I'm happy to make it work - I just don't know what looks best here.
Also should a single pkg install failure stop the rest in that command
from running?
I think I'm fine with /anything/ as playbooks only need the failed =
True/False and there is nothing to my knoweldge
that uses the finer detail yet. I'd say returning a hash of results
by each package broken out seems somewhat logical,
as long as there is still a rolled up failed/changed at the top level.
I don't plan to use this data in ansible, so it would be informational
or for API consumers -- that's why I'm saying I think
we can just do it. Ideally the apt module should do something mostly similar.
2. localinstall - I don't know what that would look like declaratively:
state=installed pkg=/some/pkg/over/here?
sounds reasonable to me.