Set-theoretic approach to host inventory - fail. Any way out?

Archives Ansible Project
1

Hello,

I'm trying to encode following host schedule into Ansible inventory:

1. There're multiple distinct services.
2. Each service is implemented by production host, staging host, and
unlimited number of (on-demand) development hosts.
3. Production and staging hosts are all distinct and mostly static.
4. Devel hosts are more interesting. They may be implemented as Vagrant
VM (== localhost) or a random EC2 instance. What's more interesting is
that any such devel host can run any service (not at the same time).

So, I;m trying to encode that in terms of Ansible host groups. Before I
even start, I consider that by real-world security constraints it's
nice to split inventory along production/development axis. So, I came
with something like (staging stuff not included to make it less
cumbersome):

=== hosts_prod ===
[service1]
service1.ip.com
[service2]
service2.ip.com
[prod:children]
service1
service2
=== hosts_devel ===
[vagrant]
localhost
[ec2-devel]
ec2-1-2-3-4.compute-1.amazonaws.com
[devel:children]
vagrant
ec2-devel
[service1:children]
devel
[service2:children]
devel

2

First question,

ansible --version ?

Second point

“I obviously won’t argue that Ansible should do hierarchical matching.”

It completely does.

I’m having difficulty drawing the lines in between your lines of questioning as you’re not showing full ansible command line executions and playbook snippets for your playbook runs.

If you can show that, with specifically which ones you think work and do not, I’d understand better.

3

If I am understanding this correctly, https://github.com/ansible/ansible/issues/4505 is what he was expecting, as I was.

4

And then I reread and, right after I sent, realize that they have similarities, but are totally different.

5

Hello,

Thanks for reply.

First question,

ansible --version ?

Initially this was tested with ansible 1.3.4 installed from Ubuntu PPA, I see that now 1.4 available via it and upgraded, but behavior is the same.

Second point

“I obviously won’t argue that Ansible should do hierarchical matching.”

It completely does.

I’m having difficulty drawing the lines in between your lines of questioning as you’re not showing full ansible command line executions and playbook snippets for your playbook runs.

If you can show that, with specifically which ones you think work and do not, I’d understand better.

Ok, I prepared this usecase as complete Ansible config: https://github.com/pfalcon/ansible-ambiguous-groups

Having there one play with “- hosts: service1” and another with “- hosts: service2”, one might expect that

“ansible-playbook -i hosts_devel -l “service1:&vagrant” site.yml” would run only play with “- hosts: service1”. However, both are run:

6

Yes, it also deals with issue of host selection, but doesn’t touch “ambiguity” of how groups are defined. Thanks for the pointer anyway.

7

"Having there one play with “- hosts: service1” and another with “- hosts: service2”, one might expect that

“ansible-playbook -i hosts_devel -l “service1:&vagrant” site.yml” would run only play with “- hosts: service1”. However, both are run:"

Nope, both plays are run, however the hosts are limited down to things in the service1 and vagrant groups.

Your localhost box is selected in the service2 play because devel has vagrant has localhost.

Working as designed.

8

Hello,

"Having there one play with "- hosts: service1" and another with "-
hosts: service2", one might expect that

"ansible-playbook -i hosts_devel -l "service1:&vagrant" site.yml"
would run only play with "- hosts: service1". However, both are run:"

Nope, both plays are run, however the hosts are limited down to
things in the service1 and vagrant groups.

Your localhost box is selected in the service2 play because devel has
vagrant has localhost.

Working as designed.

... or in other words, Ansible doesn't do "hierarchical", symbolic
matching, just reduces all symbolic expressions to host sets and
does set arithmetic on them - well, that was said in the original mail,
and I don't argue it should do it differently.

The original question still remains - how to "tell" Ansible that
"localhost:&service1" and "localhost:&service2" are two "different"
localhosts and it should discriminate tasks on "service1 vs
service2" part, not unify all tasks together based on "localhost" part?

My thinking led me to an idea that host inventory files for ambiguous
hosts (devel hosts in this case) need to be split per "service" axis
(just like I already did by development/production axis). That will get
rid of ambiguity - for each localhost, there will be a single service in
Ansible's scope. Bad news is that it's not DRY - for 100 services,
there will be need for 100 inventory files ;-(. "Good" news is that
they are all similar, so can be generated/maintained by a meta-level
script. I still wonder if there's a better solution.

9

All of this pseudo-academic confusing language is going to prevent most everyone here in holding a conversation with you.

Your conclusion that you somehow need 100 inventory files is completely strange – nobody does that.

Concepts like a “service axis” and “ambigious localhosts” are things you are creating in your own mind, they are not Ansible concepts, and I think you’re really making this all too hard.

Apologies if I’m seeing an educational empasse, but all of the above is totally not neccessary – if you can’t reset back to basic concepts and try to solve simple problems simply, perhaps it’s not for you?

10

Hello,

All of this pseudo-academic confusing language is going to prevent
most everyone here in holding a conversation with you.

Well, I'll leave "pseudo" part on your side, but configuration
management has always been subject of academic research - CFEngine
and Quattor touted that they have research and formal proofs behind
their way of organization and implementation. I'm all for KISS and
"radically simple" approach, but not for oversimplification and
random gaps in functionality.

But all in all, I started my original question
(https://groups.google.com/forum/#!topic/ansible-project/zsudA_fQshk)
with pretty simple 4-point description of what I'd like to achieve,
hope it's on its own is not confusing.

Your conclusion that you somehow need 100 inventory files is
completely strange -- nobody does that.

That's somewhat oversimplified. I made a hypothesis, that if some
services need to be independently deployed to the same host, that
would require a separate host inventory file per service. I'm not sure
that's best solution, asking for other ways to do it.

Concepts like a "service axis" and "ambigious localhosts" are things
you are creating in your own mind, they are not Ansible concepts, and
I think you're really making this all too hard.

But we both came to conclusion that standard Ansible concepts like
"host groups" don't behave in the way which would be enabling for
implementation of that 4-point host schedule. And to understand why is
that and what to do about it, it takes to introduce more
higher-level/detailed concepts. I'm still learning Ansible, and would
be glad to know better terms/approaches for it, and appreciate
following thru this and helpful hints.

Apologies if I'm seeing an educational empasse, but all of the above
is totally not neccessary -- if you can't reset back to basic
concepts and try to solve simple problems simply, perhaps it's not
for you?

Simple solution for simple problem sounds good, but what about more
complicated problems?

As for the last part, we had a pretty detailed comparison of Salt and
Ansible, and current versions of Salt lose in ease of bootstrapping for
development use (that's often overlooked with configuration management
tools!). This specific issue is also unlikely to be handled there
better - granted, it's somewhere on advanced end of ConfigManagement
stuff (so I don't expect any simple magical solution). With that, I'd
prefer to stay on objective side - what Ansible can't do, what can, and
how. What's good for people they certainly can decide on their own ;-).

11

(A) I strongly disagree with the theory that CFEngine making them complicated means things need to be made complicated, or that by in any means we must be constrained by that legacy. Making simple things sound complicated when they aren’t is not something we believe in.

(B) This list is really not the place to discuss other tool comparisons, there is a limitation in my ability to respond out of professional courtesy that limits my responses.

(C) I’m not sure where you are going with “random feature gaps”, it’s much better if you be explicit and ask “how I can do X”. It’s a problem though when we’re unable to communicate effectively – so yes, when I ask that you simplify a conversation I’m doing that because I can’t understand what you are trying to do easily.

(D) “I made a hypothesis, that if some
services need to be independently deployed to the same host, that
would require a separate host inventory file per service.”

It’s hard to tell what’s a hypothesis and what’s not, but like I said, nobody has to do that. You don’t need to define one inventory file per service ever. You might wish to have seperate playbooks or something and a master site.yml that includes multiple versions of them.