New "with_fileglob", lookup plugins system

Remember with_items? It's annoying to have to list a lot of files to
template out if they all belong in the same directory. In fact,
there are all kinds of times you may want to work
with variables that come from other sources that you don't want to
enter in. Let's make this easier.

Starting now, you can now do this:

== BEGIN EXAMPLE ==

- hosts: somehosts
  tasks:
    - file: src=/tmp/foo state=directory
    - copy: src=$item dest=/tmp/foo
      with_fileglob: /etc/*

=== END EXAMPLE ==

This would copy /etc/motd into /tmp/fuzz/motd and so on! Boom!

Developers only: "fileglob" is a lookup plugin, see
runner/lookup_plugins in the source code. As with all of our
plugins, it is user extensible. A lookup plugin takes a simple
string of terms and must return a list. Other proposed lookup
plugins might be to get a single scalar value from a web service, like
the current temperature in Siberia. It can be ANYTHING. Wait, you
say, that is a single value, and not a list! How do a write a plugin
for that? No problem, make the plugin return a list of one element!
Want your lookup plugin to return a list of hashes? No problem, it's
cool with that.

In the future, there can be other types of lookup plugins. The
aforementioned and existing "$PIPE" and "$FILE" template hacks will
also be ported to lookup plugins eventually, maybe
even super soon. In which case their syntax will remain, but in many
cases, you can do simpler stuff!

One immediate plugin that could come to mind might be this:

vars:
  users: [ 'bob', 'timmy', 'fred' ]
tasks:
  - authorized_key: user=${item.name} key=${item.key}
    with_ssh_keys: users=${users}

JP Mens originally implemented quite a few more, and I imagine many
are core shippable. I will make a note to write this one in
particular for 0.9, but that looks like an AWESOME way to distribute
authorized SSH keys to me.

Can you think of more? Sure. 'with_pipe' is an easy arbitrary one,
which is more or less like xargs for Ansible.

One note: As a result of this the 'force' option of the file module
is somewhat munged because it attempts to do crazy things like convert
between directories and files and ... in my opinion
this is sort of crazy. I will spend a bit of time looking at it and
the force= option on the file module, as I suspect hardly anyone uses
it, is likely to get scrapped.

Check it out and let me know what you think! Code was more
complicated than I imagined than to achieve this level of syntax, so
I'll warn you it's not trivial to trace how this works. Some
refactoring will come later.

--Michael

Michael DeHaan wrote:

<cool stuff snipped>

One note: As a result of this the 'force' option of the file module
is somewhat munged because it attempts to do crazy things like convert
between directories and files and ... in my opinion
this is sort of crazy. I will spend a bit of time looking at it and
the force= option on the file module, as I suspect hardly anyone uses
it, is likely to get scrapped.

I don't see how this is related? But I certainly do use it, it is a lot
cleaner than the 3 actions required otherwise to set /etc/localtime.

Daniel

I don't see how this is related? But I certainly do use it, it is a lot
cleaner than the 3 actions required otherwise to set /etc/localtime.

It may be ok, I'll take a look at it, but I had to weaken various
tests it was doing to prevent certain types of transitiions when force
was not set.

It probably was more of an issue with those checks than the actual
force behavior, I'll know when I get done figuring out which tests are
happy or not.