host patterns: order of operations

Archives Ansible Project
1

The documentation doesn’t make it clear what order of operations is applied to host patterns. It gives an example:

webservers:dbservers:!phoenix:&staging

But in my testing using similar patterns, servers from phoenix might end up matching the pattern.

Here's the rules I've figured out.

1) ":&" is a lower priority than ":".   In other words, 

a:b:&c:d is (a:b)&(c:d).   To match, the host must be in (a or b) and (c or d).

2) put your "!" rules at the back to ensure that those hosts are always excluded.

I had a pattern a:!b:&c:d.   My host was in a, b & d, and matched the pattern, which surprised me.   But if I changed it to a:&c:d:!b, it doesn't match.   a:&d:c:!b also doesn't match.

I'm not sure how "!" is applied, but putting all "!" rules at the end of the pattern has worked for me so far.

thanks,
Bryan
2

Hmm, I suspect most folks haven’t tried using !x and &y together yet, and there is probably a bug here that should be filed.

The intent was:

&x means "must absolutely be in group x

!x means "must absolutely not be in group x


I'm fine with any upgrades/changes to fix anything not correct" with this behavior,
 should someone want to tackle this, otherwise let's make sure there is a ticket open.
3

“fixing” this bug would definitely be a breaking change, and would definitely reduce capability.

We have patterns like web:db:&sf:ny in our playbook, which means “web or db in the US”. If I read your intended rules correctly, this rule would never match any NY servers, and the only way to make the rule work would be to create a group “us” in our inventory and use the pattern “web:db:&us”.

In most cases this is probably an improvement, but it will be awkward when there’s no good name for the new group.

IMO, best would probably to use the C precedence rules and allow grouping with parentheses.

But I really don’t care what the rules are as long as they’re clear, documented & tested.

thanks,
Bryan

4

Funny, I’d think as I understood the patch, “web:db:&sf:ny” would be “web or db or ny that must be in SF”.

That being said, I think that patch that added this might have had some issues.

I would definitely write it “web:db:&usa” or even “web:db” and pass into the playbook “–limit sf:ny”

I think the issue here is when this was written the parenthetical support just wasn’t planned, and I’d love for these to file the C grouping plans and do parens.

I’ll need to dig and see what’s there and think about this some, lets make sure there is a ticket tracking this.

5

It looks like the “rules” I laid out in the original post are wrong.

I had a pattern

By the rules I laid out in my OP, this would be: a & ( b|c|d ) & !e

However, I have a host that’s only in group d and it matches the pattern.

I was formerly of the opinion that we could turn the current bug into a feature by properly documenting it; this would have had the advantage of not breaking anybody who relies on the current misbehaviour.

But I’m no longer of that opinion, unless somebody can come up with a sensible set of rules for the current behaviour.

Michael, if you like I will open a bug with a patch to TestInventory.py with a set of tests. What do you want: passing tests that illustrate the oddity of current behaviour, or failing tests against the intended behaviour of “&x means must match, !x means must not match”?

In the meantime I will simplify my patterns by creating a bunch of new groups.

Bryan

6

So here is what it should do.

Regardless of how and where you put the ands, it should work like this

all the things without a prefix, it can be in any one of those

all the things with an “&” prefix, it must be in those

all the things with a “!” prefix it must NOT be in those

as such, this will yield no surprises:

if you always write them that way.

If the code doesn’t sort them and split them that way, it basically should, and that should greatly simplify things.

There is definitely no parenthethical evaluation.

7

Being somewhat new to this tool, I just want to make sure that I’m understanding this correctly.

So here is what it should do.

Regardless of how and where you put the ands, it should work like this

So, order would not matter?

all the things without a prefix, it can be in any one of those

Sounds like a boolean math “OR” (a union).

all the things with an “&” prefix, it must be in those

Sounds like a boolean math “AND” (a conjunction or intersection ).

all the things with a “!” prefix it must NOT be in those

Sounds like a boolean math “NOT” (in this case, a negation intersection).

as such, this will yield no surprises:

a:b:c:&d:&e:!f:!g

The boolean way to describe that set would be ((( a OR b OR c ) AND d ) AND e ) NOT (f OR g) if I’m reading this correctly (using no implicit priority of operations). That could also be written (((( a OR b OR c ) AND d ) AND e ) NOT f) NOT g to match the original more clearly (same value in both cases).

if you always write them that way.

Didn’t you just say order doesn’t matter?

If the code doesn’t sort them and split them that way, it basically should, and that should greatly simplify things.

There is definitely no parenthethical evaluation.

So, the order these should always be evaluated is then:
Gather the largest set (x OR y OR…),
Establish the required memberships (…AND r ),
Exclude simple sets (…NOT n Not m).

There is no provision for explicitly excluding intersections or unions of sets.

Is that the intention?

8

So, order would not matter?

If I understand correctly, order should not matter, but it does. This is
a bug.

Bryan

9

I said order should not matter. The poster seems to implying it does, and I am saying we should fix that glitch.

No, that’s not really the intention.

See my previous post for how this works.

"
all the things without a prefix, it can be in any one of those
all the things with an “&” prefix, it must be in those
all the things with a “!” prefix it must NOT be in those
"

10

Yes.

11

I said order should not matter. The poster seems to implying it does, and
I am saying we should fix that glitch.

As you created this tool I presume that you know what you're talking
about here, so no disagreement from me on that matter.

No, that's not really the intention.

I'm not sure what that comment applies to, as the answer isn't inline
with the original.

See my previous post for how this works.
"
all the things without a prefix, it can be in any one of those
all the things with an "&" prefix, it must be in those
all the things with a "!" prefix it must NOT be in those
"

And how is that different from how I transliterated your example?

In any case, the operators have some sort of precedence (as order is
deemed to not matter, kind of like Latin), which I attempted to
delimit in my original post to this subject (at the end). If I presume
that you meant to say that my conclusion was incorrect what should I
be reading to set me straight? You appear to have just said that I was
incorrect and then re-asserted the language that led me to my
conclusion.

(I understand that I'm not writing out my understanding in your quoted
grammar--that is intentional. My conversions to sets used no implicit
priority, which is why I had to use parentheses.)

12

I mean to say there is no intention of representing parenthesis like you might find in C with the and statements.

Perhaps I misunderstood your request for clarification, sorry about that if so.

What I mean to say is …

hosts: a:b:c:d:&e:&f:!g:!h

means, in set notation:

(Host matches a or b or c or d) AND (Host matches e AND f) AND (Host does not match g and does not match h)