# bash\_completion issues and some ideas **URL:** \ **Category:** Ansible Developer\ **Created:** [April 4, 2014, 9:40pm UTC](https://forum.ansible.com/t/bash-completion-issues-and-some-ideas/14945 "2014-04-04T21:40:37Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![dmitry.a.burmistrov](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@dmitry.a.burmistrov](https://forum.ansible.com/u/dmitry.a.burmistrov)\ **Post date:** [April 4, 2014, 9:40pm UTC](https://forum.ansible.com/t/bash-completion-issues-and-some-ideas/14945/1 "2014-04-04T21:40:37Z")
Hello Ansible team! I’m having great time with Ansible and I want make my experience (and Ansible) better. My story is related with administrating over 500 nodes with legacy and hacks, so I can’t do everything “The right way” (and this is just “my personal” inventory part 😉 ). That’s why I had to start thinking about extra options. I will describe a few thoughts and ask you for discussion. And my apologies for possible mistakes in my English. 🙂 So, let’s go! I am using my Ansible bash\_completion (full of hacks) and it helps me now. So I promised to share it with community. Today, I started improvement work on it… and had first blockers. I tried to make it really cool and support all possible completions. First one is inventory completion (groups, hosts). I have my inventory-script with extra feature – show all hosts and groups (I have got about 60+ groups and 450+ nodes). And that is really “The Feature” of CMS for me (and all my colleagues). Somewhere we have poor infrastructure, sometimes we have special tasks – so we use ansible cli for per host/group shell executions (no playbooks). For me it is a very big reason to use second field (host\_group\_pattern) with bash\_completion. And I failed with it today. I can’t do in my terminal this: ansible my\_cool\_group:!dead\_hosts:! [unused.example.com](http://unused.example.com) -m shell -a ‘uptime’ Bash use symbols “!”, “&” for its internal things. So we can do an “ugly hack” (this is just my opinion – no offence 🙂 ): ansible “my\_cool\_group:!dead\_hosts:! [unused.example.com](http://unused.example.com)” -m shell -a ‘uptime’ But it is (maybe) uncompletable and totally unhandy in my background/case. So let me propose you another syntax – we can replace this characters this way: - “!” =\> “-”, # this is like numbers: positive and negative 😉 - “&” =\> “^”, # conjunction symbol - “:” =\> “,” (optional). # I really feel comma for this case – we use commas everyday for lists (appendings) I _totally_ understand about changing this part in so huge project, so it could be smooth. “–future” option for this behavior for example. Think about it. My UX experience is all about this thesis. Let’s go further, second thing: in my practice we have a very good agreement about group naming – start group name with special character (I prefer “@” and someone use “%”). UX remark: it helps our vision to extremely fast distinguish hosts from groups – it is about, how our sight works (visual pattern at the first word/name symbol). I’m asking you for keeping this ability (support some “special” characters in group names) and propose to think about this idea for Ansible “Best practice” (not insisting here, just sharing thoughts). Third one! I have my hack with inventory script for getting group and host lists, but Ansible doesn’t have it for now. I think, it is a good idea to separate inventory features from Ansible part (CMS) – I am about “–list-hosts”. If we extract this to another “tool” (for example “ansible-inventory”), we can make ansible (tool) keys more focused and at the same time make Ansible (project) more powerful about inventory integration (I will write a bit more about this later in this message). Ansible-inventory can be improved by “–list-groups” (minimal required key) and other. And more! We could pass parameters to Inventory script – it would proxy any Inventory system functionality to ansible toolset. One tool – infinite abilities, homogeneous environment (as a UX guy, I’m double-pointing this point! 😉 ) Fourth. Ansible have great option “–limit”, in our practice we faced with an issue: limit hosts by variable (still in command line) – can we add this option too? # Suddenly pretty short point. 🙂 Next, fifth! Last one about my bash\_completion issue. I grab options for completion from “ansible --help” (and the same for other tools), but it would be much better to have some “contract” here – agreement about fetching list of keys. And the questions: - can I rely on current “–help” output format? - or you can prompt an existing way (maybe I missed it) Great, that’s all questions based on bash\_completion work. I think I have some more notes, but there are a bit outdated. I promise to examine them and to patches or reports. 😉 Last part, a little bit more about my background – I am going to start Inventory system project and I want to have integrations with Ansible. In my environment we really need inventory system more powerful than ini-file. We work with our current solution in a bit more complex way. And I really feel to use one toolset for close tasks. And my vision is an “ansible-inventory” “glue/proxy” tool to integrate any Inventory system with this extremely powerful CMS tools – Ansible. Thank you very much for reading, I hope we can have a very positive and fruitful discussion. I hope Michael will join this theme. ---
**Author:** ![dmitry.a.burmistrov](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@dmitry.a.burmistrov](https://forum.ansible.com/u/dmitry.a.burmistrov)\ **Post date:** [April 4, 2014, 9:49pm UTC](https://forum.ansible.com/t/bash-completion-issues-and-some-ideas/14945/2 "2014-04-04T21:49:41Z")
Omg! I forgot one big thing: I would be glad to do some work by myself, so it’s not just feature-request. I want to discuss, take an approvement and start working. ---
**Author:** ![Michael\_DeHaan1](https://avatars.discourse-cdn.com/v4/letter/m/5e9695/32.png) [@Michael\_DeHaan1](https://forum.ansible.com/u/Michael_DeHaan1)\ **Post date:** [April 5, 2014, 1:53am UTC](https://forum.ansible.com/t/bash-completion-issues-and-some-ideas/14945/3 "2014-04-05T01:53:08Z")
> Hello Ansible team! > > I'm having great time with Ansible and I want make my experience (and > Ansible) better. > My story is related with administrating over 500 nodes with legacy and > hacks, so I can't do everything "The right way" (and this is just "my > personal" inventory part 😉 ). That's why I had to start thinking about > extra options. I will describe a few thoughts and ask you for discussion. > And my apologies for possible mistakes in my English. 🙂 > > So, let's go! > I am using my Ansible bash\_completion (full of hacks) and it helps me now. > So I promised to share it with community. Today, I started improvement work > on it... and had first blockers. I tried to make it really cool and support > all possible completions. Sounds like a cool project... BTW, in case you haven't got to modules yet, check out: [https://github.com/dominis/ansible-shell](https://github.com/dominis/ansible-shell) > First one is inventory completion (groups, hosts). I have my > inventory-script with extra feature -- show all hosts and groups (I have > got about 60+ groups and 450+ nodes). And that is really "The Feature" of > CMS for me (and all my colleagues). Somewhere we have poor infrastructure, > sometimes we have special tasks -- so we use ansible cli for per host/group > shell executions (no playbooks). > For me it is a very big reason to use second field (host\_group\_pattern) > with bash\_completion. And I failed with it today. > I can't do in my terminal this: >   ansible my\_cool\_group:!dead\_hosts:!unused.example.com -m shell -a > 'uptime' > Bash use symbols "!", "&" for its internal things. So we can do an "ugly > hack" (this is just my opinion -- no offence 🙂 ): >   ansible "my\_cool\_group:!dead\_hosts:!unused.example.com" -m shell -a > 'uptime' > > But it is (maybe) uncompletable and totally unhandy in my background/case. > So let me propose you another syntax -- we can replace this characters this > way: >   - "!" =\> "-", # this is like numbers: positive and negative 😉 >   - "&" =\> "^", # conjunction symbol >   - ":" =\> "," (optional). # I really feel comma for this case -- we use > commas everyday for lists (appendings) While I can see how that helps with the above, we're not in a position where we can change things when lots of people have scripts and aliases set to the above already, not to mention playbook conventions. > I \_totally\_ understand about changing this part in so huge project, so it > could be smooth. "--future" option for this behavior for example. Think > about it. My UX experience is all about this thesis. Future seems to apply future removal, in which case we still have this problem to a degree. I want to avoid "make it configurable" being a last resort. > Let's go further, second thing: in my practice we have a very good > agreement about group naming -- start group name with special character (I > prefer "@" and someone use "%"). > UX remark: it helps our vision to extremely fast distinguish hosts from > groups -- it is about, how our sight works (visual pattern at the first > word/name symbol). > I'm asking you for keeping this ability (support some "special" characters > in group names) and propose to think about this idea for Ansible "Best > practice" (not insisting here, just sharing thoughts). Yeah this is probably all well and good if we were starting from scratch -- the @convention is a decent one. Alas, we are not, though I don't think it's terribly bad either -- because 99% of the time, folks are only matching groups (while I wouldn't write it up as such, I \*would\* consider it a best practice). > Third one! > I have my hack with inventory script for getting group and host lists, but > Ansible doesn't have it for now. I think, it is a good idea to separate > inventory features from Ansible part (CMS) -- I am about "--list-hosts". If > we extract this to another "tool" (for example "ansible-inventory"), we can > make ansible (tool) keys more focused and at the same time make Ansible > (project) more powerful about inventory integration (I will write a bit > more about this later in this message). I think a future theoretical 2.0 kinda of CLI interface would have ansible subcommands ansible show-inventory ansible playbook ansible exec ansible pull Alas, this really does need to wait... I don't want to be one to hold up progress but I'll tell you know I'd want to move our P2/P3 github queue to nearly zero before we go there and while there can and should always be a shiny future we are looking forward to, until we can get most of all the good things people have already done tested/merged/incorporated, and other needs addressed, I'm not going to prioritize this -- and the sands will likely shift underneath your pull request as we try. So what I'd probably suggest is when we \*do\* decide we want to do this, we implement that then. > Ansible-inventory can be improved by "--list-groups" (minimal required > key) and other. And more! We could pass parameters to Inventory script -- > it would proxy any Inventory system functionality to ansible toolset. One > tool -- infinite abilities, homogeneous environment (as a UX guy, I'm > double-pointing this point! 😉 ) > > Fourth. > Ansible have great option "--limit", in our practice we faced with an > issue: limit hosts by variable (still in command line) -- can we add this > option too? # Suddenly pretty short point. 🙂 It's true that Ansible doesn't have the limit by criteria features we invented in Func (actually one of our awesome GSOC interns at the time). I think there's two problems here. (A) it would need some new syntax or flag as you might want to limit by group then criteria (B) it would require the existing /usr/bin/ansible to know to run the Runner loop twice -- this wouldn't be catastrophic, but would take some doing. I've heard this request maybe 2-3 times, which is enough to make it a thing, but it's easy to do with a playbook already: - hosts: all   tasks:     - group\_by: key=family-{{ ansible\_os\_family }} - hosts: family-Debian   tasks:      - shell: echo hi And I think I'll freely admit that while /usr/bin/ansible is super useful (Runner), the focus of the app is largely going to be around playbooks -- and playbooks are why we invented things. But yes, it can be done and I think it would be nice to easily use /usr/bin/ansible to do things like ping your Debian hosts or reboot all servers where Apache is installed, without writing a playbook. I'd entertain patches for /usr/bin/ansible for this, provided they don't impact runner/\_\_init\_\_.py. > Next, fifth! > Last one about my bash\_completion issue. I grab options for completion > from "ansible --help" (and the same for other tools), but it would be much > better to have some "contract" here -- agreement about fetching list of > keys. And the questions: >   - can I rely on current "--help" output format? I'm not really willing to say "yes" here. But I'm not anticipating change. >   - or you can prompt an existing way (maybe I missed it) > > Great, that's all questions based on bash\_completion work. I think I have > some more notes, but there are a bit outdated. I promise to examine them > and to patches or reports. 😉 > > Last part, a little bit more about my background -- I am going to start > Inventory system project and I want to have integrations with Ansible. In > my environment we really need inventory system more powerful than ini-file. > We work with our current solution in a bit more complex way. And I really > feel to use one toolset for close tasks. And my vision is an > "ansible-inventory" "glue/proxy" tool to integrate any Inventory system > with this extremely powerful CMS tools -- Ansible. Excellent, the whole reason external inventory scripts in Ansible exist are to give people more options in terms of organizing how they organize [sic] -- there are a lot for various cloud providers but it can pull from anything you want. ---
**Author:** ![dmitry.a.burmistrov](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@dmitry.a.burmistrov](https://forum.ansible.com/u/dmitry.a.burmistrov)\ **Post date:** [April 5, 2014, 6:54am UTC](https://forum.ansible.com/t/bash-completion-issues-and-some-ideas/14945/4 "2014-04-05T06:54:12Z")
I will look at ansible-shell project, added to my research list. Let’s try to talk about Ansible 2.0. In big communities and companies changes are hard to move i production – it is the reality and I’m ok with it. So I guess, Ansible 2.0 would be released after 6 months since today (is this kind of true?) for example. First point – I’m not pushing my wants to Ansible 1.6. Second one – as I feel, we don’t live in “Java world” with 100% backward compatibility. And Ansible migrated to Jinja templates – I fixed all my roles and playbooks (added “{{” and “}}”), you warned me about it for a few months. It was ok. I feel this syntax changes ([:!&] → [,^-]) are the same situation. Third point – ansible and ansible-shell are good for their own cases, one tool can’t cover everything. Fourth – this changes will give benefits (yes, I understand some discomfort about migrating – and it not killing). Fifth, last one – ansible (command/tool) already have almost everything possible needed for non-playbook usage. Why not to improve it? Unified environment is much better for daily usage, so I always vote for founded unifications. It seems reasonable for me to believe this changes can be done in Ansible 2.0. Are you agree with this way? As I’m not a terrorist ( 🙂 ), I think I can do some simple completion for now and make one better for Ansible 2.0. P.S. I skipped other questions for now (I’ll return to them after this one), because I am better in focused discussions – on point in a time. I hope, it’s ok. ---
**Author:** ![Michael\_DeHaan1](https://avatars.discourse-cdn.com/v4/letter/m/5e9695/32.png) [@Michael\_DeHaan1](https://forum.ansible.com/u/Michael_DeHaan1)\ **Post date:** [April 5, 2014, 7:09pm UTC](https://forum.ansible.com/t/bash-completion-issues-and-some-ideas/14945/5 "2014-04-05T19:09:23Z")
> I will look at ansible-shell project, added to my research list. > > Let's try to talk about Ansible 2.0. In big communities and companies > changes are hard to move i production -- it is the reality and I'm ok with > it. So I guess, Ansible 2.0 would be released after 6 months since today > (is this kind of true?) for example. First point -- I'm not pushing my > wants to Ansible 1.6. There's absolutely no plan for anything like this currently. > Second one -- as I feel, we don't live in "Java world" with 100% backward > compatibility. We still fight hard for this. Nor would I agree that Java nails this well -- vendoring of jarfiles to isolate deps runs pretty rampant. I don't think Jinja2 can be used as an argument here. As Ansible was young it was finding out how things wanted to be. Supporting Jinja was one of those transformative lifestyle events. Host specifiers are working though, and simple, and we do not need to tweak them. > And Ansible migrated to Jinja templates -- I fixed all my roles and > playbooks (added "{{" and "}}"), you warned me about it for a few months. > It was ok. I feel this syntax changes ([:!&] -\> [,^-]) are the same > situation. > Third point -- ansible and ansible-shell are good for their own cases, one > tool can't cover everything. I think we do an excellent job of covering 95% of mainstream use cases and our batteries included philosophy does support this -- and prove it quite successful. Wherever possible we'll take good ideas and move them into the project. Here's a good time to quote Spock in Star Trek II -- "needs of the many, outweigh the needs of the few, or the one". > It seems reasonable for me to believe this changes can be done in Ansible > 2.0. Are you agree with this way? I don't -- and we will not be changing the way we denote host specifiers because they make a tab completion script happy. I am interested in exploring how to do dynamic limits for /usr/bin/ansible, but this should not be the "--limit" keyword since it already has meaning. It should probably be "--where". ---
**Author:** ![dmitry.a.burmistrov](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@dmitry.a.burmistrov](https://forum.ansible.com/u/dmitry.a.burmistrov)\ **Post date:** [April 5, 2014, 7:57pm UTC](https://forum.ansible.com/t/bash-completion-issues-and-some-ideas/14945/6 "2014-04-05T19:57:23Z")
I am a bit lost – can you formulate your vision of ansible command? I think, I don’t get it in your way. Here are questions, I’m worrying about: - is it (ansible command script) an important part of ansible toolset or just small script for fill some breach? - should people use ansible command in regular work? (it is about “how often people use is” – from almost never to every day) I attached bash\_completion file – try to use it (it is a bit outdated, but still works ok for test case). About “–where” (“–limit”, as you like): I haven’t discovered script, so I’ll just try to offer an example of syntax structure just as one of possible ideas: (–where host:expressions), (–where group:expressions), (–where vars:path.to.var.name::expression). > **(attachments)** > > [ansible.gz](https://forum.ansible.com/uploads/short-url/lN7ZAcIzsHDc3wydkeWcHqL75p9.gz) (523 Bytes) ---
**Author:** ![dmitry.a.burmistrov](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@dmitry.a.burmistrov](https://forum.ansible.com/u/dmitry.a.burmistrov)\ **Post date:** [April 6, 2014, 6:18pm UTC](https://forum.ansible.com/t/bash-completion-issues-and-some-ideas/14945/7 "2014-04-06T18:18:15Z")
Strange, I can’t see your last message in GG (only in mail). I agree with you about --where. Great! About syntax operators – I rewrote my points: - [!&] are pretty dangerous symbols for bash (I follow Raskin and others in UX-questions) - my guess: people use non-“:” operator rarely - current documentation about them generates user mistakes (no quotes in examples) - this could make new users unhappy for that moments (when bash writes errors after “trying correct examples from docs”) - people are not robots and can do mistakes (I do them sometimes 🙂 ) – anyone can forget quotes - people do more errors under stress (10 min downtime in my service costs about 10 000$, so I write commands quick, sometimes with errors) – this make: pain = pain\*2 - I and somebody else can be unhappy, when bash showing errors caused by “!”, “&” instead of doing real job - my group names are longer, than 15 characters – it is really hard to live without completion Can we _add_ “,”, “^”, “-” operators to current list and update doc part about Patterns? Everything would still work. But no one will do mistake (if he follow new syntax). No mistakes – no unhappiness/sad. Advanced bash\_competion for comfortable work. You made my worklife better, I shared my experience with other people. Maybe I am in 5-10% users, who use Ansible for not only common cases. But we want improvements for us too, we want be happier and share our experience with more success-stories. So adding new operators will decrease user mistakes (UX will be better), user comfort will grow. Happy user is good for project. Does it sound better now? ---
**Author:** ![Michael\_DeHaan1](https://avatars.discourse-cdn.com/v4/letter/m/5e9695/32.png) [@Michael\_DeHaan1](https://forum.ansible.com/u/Michael_DeHaan1)\ **Post date:** [April 6, 2014, 7:52pm UTC](https://forum.ansible.com/t/bash-completion-issues-and-some-ideas/14945/8 "2014-04-06T19:52:07Z")
> Strange, I can't see your last message in GG (only in mail). I agree with > you about --where. Great! > > About syntax operators -- I rewrote my points: >   \* [!&] are pretty dangerous symbols for bash (I follow Raskin and others > in UX-questions) When quoted they don't go to bash. >   \* my guess: people use non-":" operator rarely They are used a fair amount in playbooks. /usr/bin/ansible also gets a decent amount of usage. >   \* current documentation about them generates user mistakes (no quotes in > examples) Let's fix those. > Can we \_add\_ ",", "^", "-" operators to current list and update doc part > about Patterns? Everything would still work. But no one will do mistake (if > he follow new syntax). not interested, sorry. ---
**Author:** ![dmitry.a.burmistrov](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@dmitry.a.burmistrov](https://forum.ansible.com/u/dmitry.a.burmistrov)\ **Post date:** [April 6, 2014, 9:35pm UTC](https://forum.ansible.com/t/bash-completion-issues-and-some-ideas/14945/9 "2014-04-06T21:35:44Z")
Can I buy this feature? ---
**Author:** ![dmitry.a.burmistrov](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@dmitry.a.burmistrov](https://forum.ansible.com/u/dmitry.a.burmistrov)\ **Post date:** [April 7, 2014, 12:27pm UTC](https://forum.ansible.com/t/bash-completion-issues-and-some-ideas/14945/10 "2014-04-07T12:27:11Z")
Have I understood you right? My potential pull-request about [&!:] would be rejected? ---
**Author:** ![Michael\_DeHaan1](https://avatars.discourse-cdn.com/v4/letter/m/5e9695/32.png) [@Michael\_DeHaan1](https://forum.ansible.com/u/Michael_DeHaan1)\ **Post date:** [April 7, 2014, 4:17pm UTC](https://forum.ansible.com/t/bash-completion-issues-and-some-ideas/14945/11 "2014-04-07T16:17:23Z")
We will not be changing the way “&” and “!” work, or adding alternative syntax.