The link on ansible.cc resources page to “Share playbooks, modules, scripts, and more” and the link in the Additional Contrib Modules section of the docs are broken. What is the prescribed method for sharing playbooks, etc?
Thanks,
Jeremiah
One solution can be to cerate an appropriate repo in ansible project, but maintenance will be really complicated (and will be better have different repo for each playbook, similar to chef-cookbooks), so repo explosion can be unmanagable.
Second solution (what I prefer) can be that each user can create a project with standard name (ansible-playbook-XXXX, eg: ansible-playbook-mysql, ansible-playbook-solr, …) and all this repo will be linked in docs page. Can I suggest to create a view of all repos with ansible-playbook in repo name using Github API¹ (orded by name, score, …). Why? Information are always update (no extra-work for docs mantainers), if a user create a new repo this will be visible in list without moderation. The score of repository can be used to “ensure” a the playbook their “reliability”.
----8<-------8<-------8<-------8<-------8<-------8<-------8<—
#!/usr/bin/python
import json
import urllib2
repos = json.load(urllib2.urlopen(“https://api.github.com/legacy/repos/search/ansible-playbook”))
data = {}
for repo in repos[‘repositories’]:
if repo[‘name’].startswith(u"ansible-playbook"):
val = ‘http://github.com/’ + repo[‘username’] + ‘/’ + repo[‘name’]
data[val] = val
print data
----8<-------8<-------8<-------8<-------8<-------8<-------8<—
Yes I agree they should be developed in separate repos. I was going to prefix mine with just “ansible-” in similar fashion to my chef cookbooks “chef-” rather than using the longer prefix “ansible-playbook-” or “chef-cookbook-” but if majority tend to desire the longer then I’m happy to conform. Most of all I wanted to point out the broken links and ask what would be replacing the missing page. I’m used to the chef community cookbooks site where people can make their cookbooks more easily visible. I’m just not used to searching github but I like the idea of the dynamic search view you described. Especially sorting by score to float more valuable playbooks to the top. Thanks for that tip.
Jeremiah
Thanks for the code but what do I do with it? Where do I put it?
"Yes I agree they should be developed in separate repos."
Playbooks don't need to be done in seperate repos IMHO -- this will be
total overkill for many users. I do recommend that you have at least
one git repo.
You really shouldn't look at playbooks as if they were Puppet modules,
they are really just simple collections of files, and you can organize
them how you like.
Given, I'll admit we don't have a lot of easily shareable community
content, but I think that's ok. The vast majority of content on the
Puppet Forge of Chef sites are specific to particular operating
systems or setups, or ways of managing things, that as soon as you
want to use them, you have to do a lot of work to adapt. They don't
really work. As such, we concentrate on making Ansible's language
super easy to use, with the knowledge that you don't need a Forge to
crib off of, and making it much easier to design your content the way
you want to.
--Michael
Ironically, the contrib page need not link to the "share..." page,
because http://ansible.cc/docs/contrib.html is now the share page 
I'll make a note to look into the links.
Let me better explain my point of view.
Soomeone can create a playbook to configure a master slave configuration on MySQL and share their playbook. This playbook can use some variable to define some settings.
Another user can add this repo as git submodule and use this playbook without rewrite all configuration required to configure master slave MySQL cluster. The same user can rewuire a multinode ApacheSolr service, and can add another git submodule from other repo… etc.
This can help user to start with complex configuration in ansible without “reinvent the wheel”.
Well, I also agree they don’t need to be done in separate repos. From my experience there are more benefits when using separate repos if you intend to share the playbooks. I think it encourages people to reuse and contribute resulting in well designed playbooks. Why would someone want to end up cluttering themselves by having irrelevant playbooks included when they clone the repos containing the playbooks they desire?
Forgive me if my comparison with Chef is just misplaced but I saw how some months ago Opscode finally realized that housing 118 cookbooks under one github repo was just not a good way to manage them so they split them all into separate repos and there was much praise for the resulting workflow. Granted most ansible users won’t have nearly that many but still similar difficulties can emerge with just a few playbooks in a repo. I understand not having an ansible managed/approved list of playbooks but I don’t understand why it’s not a better idea to use separate repos when intending to share.
Given, I’ll admit we don’t have a lot of easily shareable community
content, but I think that’s ok. The vast majority of content on the
Puppet Forge of Chef sites are specific to particular operating
systems or setups, or ways of managing things, that as soon as you
want to use them, you have to do a lot of work to adapt. They don’t
really work. As such, we concentrate on making Ansible’s language
super easy to use, with the knowledge that you don’t need a Forge to
crib off of, and making it much easier to design your content the way
you want to.
I totally agree that users should be able to easily do things the way they want and I praise ansible for making that possible. But from a DRY perspective it doesn’t make any sense to me to practically dissuade users from easily sharing content. In the last 8 months of my time with Chef cookbooks I didn’t experience much of “They don’t really work” that you mention. The fact that I could so easily get going by “cribbing off” of well designed cookbooks just made my job simpler. The fact that I could also contribute back to those cookbooks when needed not only benefited me but hopefully other users too. Why reinvent the wheel especially when someone smarter or more experienced than me has done a better or more complete job. Rather than spend days making sure I’ve hand designed a great mysql-server playbook I could just grab a reliable one from someone else and contribute back to it as necessary. It seems to me that some consideration for the sharing of content would only enhance what is already a great project.
Thanks,
Jeremiah
Not really sure what you want me to say.
I believe a lot of people's content on the various Forgey sites are of
dubious quality and the idea that there's one "Tomcat" playbook
everybody copies from is not appropriate, and then you get into a
whole enormous hassle with dependencies.
I guess I was hoping to hear that you recognized the value of sharing of and contributing to playbooks. I do understand you don’t prefer sharing because of having to manage dependencies. Dependencies were one of my concerns as I was trying to understand how best to share playbooks. I’m at least glad to flesh out the ansible share model, or lack of one, early in my use.
My only remaining question is a sincere one. I must be just overlooking something. Aside from dependency issues why do you think it’s not appropriate to have “everybody” who prefers to copy one Tomcat playbook?
Thanks,
Jeremiah
My only remaining question is a sincere one. I must be just overlooking
something. Aside from dependency issues why do you think it's not
appropriate to have "everybody" who prefers to copy one Tomcat playbook?
I was around when were desigining Puppet Forge. The problem is,
people ended up using all sorts of little quirks and modules. Debian
and CentOS Apache trees are managed quite differently, etc.
Some people use vhosts, some don't.
Some people want to front a service with nginx, some with Tomcat, etc.
Some people's templates contain modules you don't want to use.
Ultimately you really have to understand your tools in the end -- and
usually have to derive anything non-trivial.
I think the Forge idea is GREAT marketing, and great reference, but
you almost always have to adapt these things to your environment.
And that tends to imply some concept of "shadowing" or "inheriting"
from those configurations.
I encourage everyone to publish their content, and share, an probably
link the good stuff to ansible.cc/docs/contrib.html and so on so there
are good examples of larger Ansible deployments. You'll also see,
quickly, that any good site deployment beyond demo-ware becomes
non-shareable without major simplifications.
--Michael
+1
Ultimately you really have to understand your tools in the end – and
usually have to derive anything non-trivial.
I think the Forge idea is GREAT marketing, and great reference, but
you almost always have to adapt these things to your environment.
And that tends to imply some concept of “shadowing” or “inheriting”
from those configurations.
Thanks, those are good points. I’ve seen a lot of power and effectiveness from the Chef cookbooks I’ve used but I also can see where things can run astray. It’s helpful for me to know up front that I’m better off just developing my own playbooks. While it will take more time building from scratch at least the complexity of dependency resolution is avoided and I’ll also know that I’ll have complete control of as well as responsibility of the final result. Thanks for your insights.
Jeremiah
* Michael DeHaan <michael.dehaan at gmail.com> [2012/11/02 10:41]:
I was around when were desigining Puppet Forge. The problem is,
people ended up using all sorts of little quirks and modules.
I'd like to comment on this point. We are heavily invested in puppet
and when puppetforge came out, we were understandably very excited
about it. However, after spending time with it, we literally found
not a single module on it that we could reuse without significant
modification, due to these "quirks" (which is a very kind way of
describing the problems we saw with many of the modules!). This
might say more about the idiosyncrasies of our environment, which is
almost pathologically heterogeneous (we still have OS/2 Warp and
PDP-11's in production, for fuck's sake), but at best, puppetforge
serves as a collection of patterns and anti-patterns for us to
emulate, integrate, or avoid in our own modules.
So in the end, the most frustating thing about puppetforge was that
it was a huge tease followed by a crushing let-down, since it is
marketed as a CPAN- or pip-style archive of reusable libraries.
I would hate for an ansibleforge (or whatever) to end up in the same
position.
Yep. This is why I really haven't invested in trying to build one.
There are things we can do (this, dry run modes*, windows support**),
etc, that seem really good for adoption (and would be, no doubt), but
don't really deliver on what they say they provide.
So we don't do those things.
* = because they never account for dependencies, side effects, rpm or
dpkg scripts, rpm dependencies, config file errors, version updates,
failures of services to restart, sharing of variables between nodes,
and all the things that are most likely to happen. At most, it's a
*bit* of information about what might happen, but it's not all of it.
I've shared my thoughts on that before though.
** = the way Windows admins want to manage Windows systems is
completely different than the way Linux admins think Windows admins
want to manage Windows systems, and most Microsoft-folks seem revolted
about the idea of installing a Python/Ruby stack on a Windows box.
I find the whole idea of 'strategybooks' confusing.
I guess while writing these playbooks I’ve been repeating lots of some tasks over and over. So I’ve been toying with this idea with prototypes of first finding what segments are repeatable and trying to make them into simple building blocks so you don’t have to start off an ansible project from scratch.
Just like puppet, I’ll keep refactoring and updating the docs until I’ve settled one that best fits our scenario with mult-datacenter, multi-admins, and devs that want to have some control in production tuning on their apps. With puppet I essentially have the same layout with a t top level generic modules. Then if you really wanted to tune further, you can simply extend a class and override the base. Then you can create logical roles that include a selection of either your base modules and or extended ones. My jr admins can then build server roles by using that as a base. They wouldn’t need to know puppet of the bat except how to assemble the modules I’ve built for them and map it to our provisioning naming scheme. Cobbler then kickstarts a new instance and they are off playing with their new server role.
But like I said this is my first iteration or experiment with ansible, so I wanted to poke for ideas and would really like a solid framework for reusabilty. And yes I’ve looked at and use includes heavily. Maybe I’m just over engineering for reuse. Perhaps I just need more advance playbook examples to better grasp your vision on reuse.
I like ansible for not requiring agents installed on clients and it’s freedom of language usage. I would also love to migrate my existing workflow that has been working well so for our jr admins and developers.
I think, generally speaking, trying to craft Puppet/Chef/other style
architectures in ansible will result in something that feels weird.
This is kind of what this feels like to me. Basically when I built
ansible, the idea was to make things maximally auditable, and this
means generally flat, and non-clever. This means it's actually
easier to leave (stop using) ansible than most tools. At least I
hope that is the case.
You are certaintly free to build whatever you want, but I don't think
any idea of a 'community playbook' should really exist, because what
happens, is different people have different ideas about levels of
abstractions or complexity provided, and in general, these show bad
examples of ansible looking like an architecture it wasn't
neccessarily trying to look like. Ansible is really about everyone
understanding the atoms and being able to write their own bits, and
not getting too attracted to perfection.
You should basically be able to knock some basic things out *fast*,
and get on to some other work, rather than getting folks really
enthused about using the tool. It wants to get out of the way, and to
do that, you want to keep things extra simple.
I think if you try for that, that workflow you think your junior
admins like, they might actually like it a lot more.
In other words, I think we have been seduced by our tooling to aspire
to have complication and order (and we then start to really enjoy
organizing things), and really what we need most is the most minimal
thing possible that just gets work done.
I can definitely understand the above. The biggest issue I've ran into with puppet in the past is that you end up with extremely clever recipes which are impossible to modify w/o breaking a dozen other hosts' expectation. A good reason to try and keep it as simple as possible in my ansible playbooks, I think.
-sv