So, we've been talking about speed. I have an interesting idea that
I think is quite awesome and probably negates some of the other
discussion. I call this fireball mode.
One of the core tenets of ansible is no daemons and no bootstrapping.
Suppose we bend that rule for this particular mode. We allow
TEMPORARY daemons, and self bootstrapping. Not terrible, right, if
ansible can bring itself up?
Imagine this playbook:
Fireball mode is a huge success. It is still in very much a
development state, but initial tests SSH'ing to 127.0.0.1 yield a
performance increase of 8x.
Here is what it looks like. What follows is a two-step play that
first sets up the fireball daemon module. Super easy to do, you will
typically do this for all hosts in your infrastructure in a massive
parallel sweep.
Then, once that is done, you can use fireball just like any other
connection type.
Here's the commit: (again, rough proof of concept, more to do):
https://github.com/ansible/ansible/commit/69612ba16d1985dd63efa07f40df895813e65ecb
Blog post:
http://michaeldehaan.net/post/32378722265/ansible-learns-to-fly-0mq-that-sets-up-itself
Sure.
If your cluster is situated oversees, behind firewalls, through so
many hops, etc, you should install ansible on a local cluster node.
Then you can use -c ssh, paramiko, or fireball from there.
Just log in to your management node on your cluster first.
Thank you for suggestion.
In such case I have to use that management node as a “trampoline” or jump host, cause I need to deploy several files from my local host to the cluster ones.
Probably on my local machine I have to run one playbook which copies files to the management host and then it starts another playbook execution on remote end with help of shell module.
Or can you suggest some better approach?
BTW, is such idea worth something, when ansible natively supports such “jump host” mode with fireball mode in cooperation?
In such way the firewall problem could be overcome.
Regards,
Igor.
Thank you for suggestion.
In such case I have to use that management node as a "trampoline" or jump
host, cause I need to deploy several files from my local host to the cluster
ones.
Probably on my local machine I have to run one playbook which copies files
to the management host and then it starts another playbook execution on
remote end with help of shell module.
Or can you suggest some better approach?
Two playbooks.
One to push the content to the management node (or just do a git
checkout, see below)
Another that you run ON the management node.
Keep them both in the same git repo to keep it simple to manage on your end.
Sure you could trigger ansible with ansible.
Another option is to just let it hop-hop up to trigger ansible-pull or
just use ansible-pull on cron.
BTW, is such idea worth something, when ansible natively supports such "jump
host" mode with fireball mode in cooperation?
Something could probably be set up zeromq to forward things along --
it would be something someone else needs to implement though as I'm a
bit too busy with other things to chase ATM.
Looks like a great example of Ansible possibilities.
But if you want some feedback...
I tried to use Salt [1] for configuration management. It is very similar
to Ansible and written in Python too. But sometimes it does not work.
Just hang without any debug output or don't connect to some hosts. I can
not find references, but other people had troubles too and were able to
trace this down to zeromq, but not resolve. I don't understand zeromq
well and I'm happy that Ansible use plain simple SSH.
Also keyczar looks like a good library, but usually it is not wise to
try to mess with crypto. Again I would prefer to stick with old good SSH.
Though, I would like to repeat, idea is very interesting.
I suspect we're not using the same Python 0mq library. As the other
app is packaged in one particular place that doesn't have the library
I'm using. Even so, we're not coded in the same way. If we do,
there's no reason the transport couldn't be switched to Rabbit later
if we really wanted.
Anyway, SSH is going no where. I love SSH. Great bootstrapping
qualities, super secure, runs as you. SSH is awesome. This is a
"because it was doable thing", and I had some insight on how to make
it more manageable.
That being said, I know of that application, the author used to be a
Func user and has borrowed a ton of ideas from it and also a ton from
Puppet. Wish it gave credit, but ah well. I'd rather not discuss it
here.
To be clear -- We are using the 0mq fireball accelerator thing in
quite different ways -- you can run it as non-root, if you want, and
it's transferred over SSH so you don't need to fiddle with
Puppet-style keysigning and cert issues, and it self-destructs after
30 minutes so it does not consume resources. All of these decisions
made after years of dealing with them in Puppet and Func architectures
-- I'm not going to do it wrong again 
There really are no
similarities other than common origin.
--Michael