Hi,
As discussed with Michael on irc yesterday, I suspected something to be wrong with the calculated group depth, which is AFAICS used to have the right variable precedence according to the depth of a group in the parent-child tree.
As I’m looking into making a custom inventory script, tailored to our very strict standardized environment, I wanted to have a way to more or less visualize the group tree, and be able to check it on possible errors.
To test this (and also to troubleshoot other inventory related stuff) I wrote an “ansible-vars” script based on bin/ansible that parses the inventory, and can show a (tree) list of groups, and a list of hosts with all of their variables. It’s not super clean, but gets the job done for now. (If interested, you can check it out here: https://github.com/ansible-contrib/ansible-plugins/blob/devel/bin/ansible-vars )
It took me some time to figure this out, as the wrong depth values I got were inconsistent
- depended on the level of subdirectories in which the ini files resided
- depended on the order of groups as noted in the ini files
As it turned out, those were two similar, but different bugs in group.y and dir.py
Example:
I’m not sure the internal variables in Ansible being weird for you are a symptom. What’s actually the problem?
As nicely as possible, please keep “TLDR” in mind a bit 
As I said earlier, the effect on variable precedence can be a corner case.
I started assembling a test inventory with some variables, and managed to
find a setup where the problem shows.
Inventory file, used commands, the expected result and the (wrong) command
output can be found in this gist:
https://gist.github.com/sergevanginderachter/5605645
one example:
hosts file contains groups with this structure; all the app* groups have a
variable set to the name of the group
. site
.. apache
... app2
.... app2-dev --> these level groups contain the nodes
.... app2-prod
.. tomcat
... app1
.... app1-dev
.... app1-prod
running a debug task showing that var:
app1-dev-node | success >> {"msg": "var=app1"}
app1-prod-node | success >> {"msg": "var=app1-prod"}
app2-dev-node | success >> {"msg": "var=app2-dev"}
app2-prod-node | success >> {"msg": "var=app2"}
So some nodes get their var from app1/2 instead of the subgroup with the
environment.
- pointing the inventory to the hosts file directly, or the parent dir also
yields different results
- restructuring the order in the ini hosts file can also influence this
- in the mean time I discovered that the patch I made does not allways
solve the problem
More details in the gist.
Serge
Please forgive my lack of brain processing power.
So if I understand this correctly there are variables defined in the tomcat subtree and variables in the apache subtree, and you’d like them to override in some way? Or which two groups are you not seeing variables blended like you like from?
This is why I strongly encourage the use of group_vars/ to put variables in, makes inventory much easier to read.
So if I understand this correctly there are variables defined in the
tomcat subtree and variables in the apache subtree, and you'd like them to
override in some way? Or which two groups are you not seeing variables
blended like you like from?
To keep things simple, just look that part of my example where the variable
'var' is defined on the app1 and app2 group, and on their respective
subgroups. The four '-node' hosts are only defined in those subgroups, and
only on 1 single place, so there is no overlap, except for parent and
child groups 'app*'
Basically, I expect to be able to define var on the app1/2 groups, as a
default, and to override them in the childgroups app1/2-dev/prod (in which
the nodes are directly defined, and have the greatest depth.
This is how precedence is documented: "" If a variable is defined in
multiple groups and one group is a child of the other, the child group
variable will override the variable set in the parent. ""
My tests show that this doesn't allways happen.
This is why I strongly encourage the use of group_vars/<groupname> to put
variables in, makes inventory much easier to read.
I normally don't use variables defined in ini files, just did that now now
for the sake of giving a complete inventory file. to test.
Serge
"To keep things simple, just look that part of my example "
I’m sorry, your example is not simple.
This is why it is very important to submit minimal examples and work things down to the smallest possible question.
Just define as few hosts and groups and variables as you possibly can to show this, and pastebin a gist of that.
If it can be done with one variable and three groups, show me that.
I'm sorry, your example is not simple.
The problem isn't simple also.
This is why it is very important to submit *minimal* examples and work
things down to the smallest possible question.
Just define as few hosts and groups and variables as you possibly can to
show this, and pastebin a gist of that.
I already tried that, it's not easy to find good examples of showing a
problem that only affects in certain conditions, in more complex
environments.
If it can be done with one variable and three groups, show me that.
My example does almost that, it only needs some other groups to trigger
the problem/
I might find an example with one variable, defined on two or three group
levels, and having some extra groups where no variable is defined. I'll
look into that tonight or tomorrow.
Serge
Maybe you can draw up some ascii art that would help me understand better, something like this:
http://pastebin.com/PYm47hcu
?
Sorry, corrected that a bit, this should be more clear for what I am looking for trying to understand:
http://pastebin.com/vzUrW6Ha
I managed to dumb it further down. This example doesn’t show all the possible effects when using the ‘hosts’ ini file directly as the inventory, but it does show when putting the hosts file in a subdirectory
Keep in mind though that the problem can show up in more complex examples, when only using an ini file as inventory. I should be able to make a more complex example to show that.
So I made a simple ini file, also with a small graph of the group tree, containing 2 levels of groups, 2 parent groups with each 2 child groups, 1 node per child group, 4 nodes in total.
One single variable ‘var’ is defined on each of the groups. Each node is expected to get the variable of the respective child group it is the sole member of.
O
utput shows thatsome
nodes get the variable from the parent group, in this case when pointing the inventory to a dir containing the ini file.
Michael,
Did you have the time to look into this yet?
Do you need more information?
Serge
Haven’t had any time yet, no.
Please copy this into a github ticket so we don’t lose it.