# pass --limit to inventory to improve speed and remove unnecessary variables **URL:** \ **Category:** Ansible Developer\ **Created:** [September 7, 2015, 11:39am UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198 "2015-09-07T11:39:40Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![Edward\_Zambrano](https://avatars.discourse-cdn.com/v4/letter/e/4da419/32.png) [@Edward\_Zambrano](https://forum.ansible.com/u/Edward_Zambrano)\ **Post date:** [September 7, 2015, 11:39am UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/1 "2015-09-07T11:39:40Z")
Hello! Our current situation: -We have a python script that builds the inventory with a bunch of variables per each group, we can notice that the bigger the JSON the inventory generates, the slower ansible runs (about 60 seconds with the full size) -usually we use --limit and run ansible against just one server, but still the inventory generates the JSON for all of the hosts (and groups) We would like to pass the --limit argument to the inventory script so we can only generate the inventory for the hosts we are running ansible against. I would like to know what do you think of this idea and if you like it I can work on the PR cheers ---
**Author:** ![Hagai\_Kariti](https://avatars.discourse-cdn.com/v4/letter/h/4491bb/32.png) [@Hagai\_Kariti](https://forum.ansible.com/u/Hagai_Kariti)\ **Post date:** [September 7, 2015, 12:57pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/2 "2015-09-07T12:57:39Z")
I would like to remind of the other usecase, where having all of the basic inventory variables available allows you to access them using the hostvars variable, even when running on only specific hosts using --limit. Always passing --limit to the inventory script will make this impossible. ---
**Author:** ![Will\_Otterburn](https://avatars.discourse-cdn.com/v4/letter/w/54ee81/32.png) [@Will\_Otterburn](https://forum.ansible.com/u/Will_Otterburn)\ **Post date:** [September 7, 2015, 2:38pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/3 "2015-09-07T14:38:58Z")
Another advantage to using limit in this way would mean that you don’t have to decrypt variables for groups you aren’t deploying to when using vault. Though this could potentially cause issues with delegate\_to as that host might not be in the initial limit. ---
**Author:** ![sivel](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sivel/32/1198_2.png) [@sivel](https://forum.ansible.com/u/sivel)\ **Post date:** [September 7, 2015, 3:52pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/4 "2015-09-07T15:52:56Z")
The biggest problem is that the limit spec is “complicated” and without just reusing the functionality in your inventory, since that isn’t likely to speed things up, the API or whatever your script queries for data would also have to support it. I know this shouldn’t really be the concern of Ansible to decide, but for other than the most simple case, I can’t imagine it will be very useful without you doing some heavy lifting to actually make it useful. We actually have a wrapper application that intercepts the limit and passes it to the inventory script by way of an environment variable, which is pretty much suiting our needs currently. Although it is another step, you can implement this behavior now by use of env vars. although potentially duplicating input. ---
**Author:** ![esco\_real](https://avatars.discourse-cdn.com/v4/letter/e/c89c15/32.png) [@esco\_real](https://forum.ansible.com/u/esco_real)\ **Post date:** [September 7, 2015, 7:26pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/5 "2015-09-07T19:26:31Z")
I think there should be an option for dynamic inventory scripts, too. At the moment if a script doesn’t provide the vars for all hosts with “–list” in the “\_meta” section the script gets called for all hosts with “–host”. [http://docs.ansible.com/ansible/developing\_inventory.html#tuning-the-external-inventory-script](http://docs.ansible.com/ansible/developing_inventory.html#tuning-the-external-inventory-script) Maybe there could be an option to pass the “–limit” to the inventory collection, too (–limit-inventory). Or maybe to implement an inventory limit (–limit-inventory )? esco ---
**Author:** ![Rene\_Moser](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@Rene\_Moser](https://forum.ansible.com/u/Rene_Moser)\ **Post date:** [September 8, 2015, 6:01pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/6 "2015-09-08T18:01:27Z")
Hi > We would like to pass the --limit argument to the inventory script so we > can only generate the inventory for the hosts we are running ansible > against. I would like to know what do you think of this idea and if you > like it I can work on the PR This would break backwards compatibility because the host variable can also be used in templates (which is damn powerful) e.g. {% for host in groups['all'] %}    {{ hostvars[host]['your\_variable'] }} {% endfor %} or in tasks - debug: msg="{{ item }} having var {{ hostvars[item ['your\_varibable\_name'] }}"   with\_items: groups['all'] ---
**Author:** ![esco\_real](https://avatars.discourse-cdn.com/v4/letter/e/c89c15/32.png) [@esco\_real](https://forum.ansible.com/u/esco_real)\ **Post date:** [September 8, 2015, 7:39pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/7 "2015-09-08T19:39:27Z")
Hello René, I think everyone is aware of this and making this “inventory limit” default isn’t an option. But if you have a huge generated inventory from a CMDB with a lot of locations, this would make really sense. For example if you want to configure some hosts in DMZ 1 at location A you wouldn’t need all hostvars loaded from all locations and zones… Maybe another option would be to load hostvars dynamically when needed. esco ---
**Author:** ![Jesse\_Keating](https://avatars.discourse-cdn.com/v4/letter/j/3bc359/32.png) [@Jesse\_Keating](https://forum.ansible.com/u/Jesse_Keating)\ **Post date:** [September 8, 2015, 7:42pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/8 "2015-09-08T19:42:18Z")
Given the number of cases where this could go awry, I believe the best way is to let each plugin implement the support for this kind of limiting, much how Matt Martz has described. ---
**Author:** ![Hagai\_Kariti](https://avatars.discourse-cdn.com/v4/letter/h/4491bb/32.png) [@Hagai\_Kariti](https://forum.ansible.com/u/Hagai_Kariti)\ **Post date:** [September 8, 2015, 8:30pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/9 "2015-09-08T20:30:18Z")
Lazy loading of variables is already supported isnt it? The plugin should just not provide the \_meta item and handle being run with the --host argument. Maybe plugins should support doing that based on config option/env variable. ---
**Author:** ![Brian\_Coca](https://avatars.discourse-cdn.com/v4/letter/b/6de8d8/32.png) [@Brian\_Coca](https://forum.ansible.com/u/Brian_Coca)\ **Post date:** [September 8, 2015, 8:39pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/10 "2015-09-08T20:39:08Z")
hostvars is already lazy, the issue is that it relies on inventory hosts, which is not lazy. ---
**Author:** ![esco\_real](https://avatars.discourse-cdn.com/v4/letter/e/c89c15/32.png) [@esco\_real](https://forum.ansible.com/u/esco_real)\ **Post date:** [September 9, 2015, 10:57pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/11 "2015-09-09T22:57:34Z")
Hello Brian, since when (maybe in 2)? With 1.9.3 this doesn’t seem to be lazy… Short example: #cat hosts `host[1:3] [ansible] ansible ansible_ssh_host=localhost` #cat test.sh ` #!/bin/bash case “$1” in –list) echo {} ;; –host) echo {"foo": "bar"} sleep 1 ;; esac ` #time ansible -m debug ansible ` ansible | success \>\> { “msg”: “Hello world!” } real 0m4.265s user 0m0.190s sys 0m0.078s ` esco ---
**Author:** ![Brian\_Coca](https://avatars.discourse-cdn.com/v4/letter/b/6de8d8/32.png) [@Brian\_Coca](https://forum.ansible.com/u/Brian_Coca)\ **Post date:** [September 10, 2015, 12:58pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/12 "2015-09-10T12:58:54Z")
@esco, not sure what you are trying to show here, as I said, inventory is not lazy, but hostvars are, hostvars are populated partially from inventory, but it is not the only source. ---
**Author:** ![esco\_real](https://avatars.discourse-cdn.com/v4/letter/e/c89c15/32.png) [@esco\_real](https://forum.ansible.com/u/esco_real)\ **Post date:** [September 10, 2015, 6:39pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/13 "2015-09-10T18:39:36Z")
In the example I only run Ansible to “debug” hosts matching the expression “ansible”. But it runs the script “test.sh” for every host (host1 - host3 and ansible: sleep 1 in the script when called with “–host” and run time 4 seconds…). So if you have a script that returns vars for hosts (with --host) every time Ansible runs this script gets called for every host in the inventory sequentially. But I think it would be nice if Ansible would call the inventory script with “–host” only when the hostvars are needed (and maybe in parallel). That’s what I would call “lazy” in this case. esco ---
**Author:** ![Brian\_Coca](https://avatars.discourse-cdn.com/v4/letter/b/6de8d8/32.png) [@Brian\_Coca](https://forum.ansible.com/u/Brian_Coca)\ **Post date:** [September 10, 2015, 8:54pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/14 "2015-09-10T20:54:04Z")
again, you are confusing inventory with hostvars (which includes inventory and more), also there is a workaround for this in inventory scripts, use the \_meta facility to populate all host variables at once and it will avoid calling the script over and over. ---
**Author:** ![esco\_real](https://avatars.discourse-cdn.com/v4/letter/e/c89c15/32.png) [@esco\_real](https://forum.ansible.com/u/esco_real)\ **Post date:** [September 10, 2015, 9:33pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/15 "2015-09-10T21:33:29Z")
Hello Brian, Yes, I am aware of the “workaround” (some posts before…). And I thought hostvars are the vars from the inventory. So yes, I am bit confused with this. Thought that only facts (terminology from the documentation but in fact that are “vars”, too) for the host are loaded lazy. Or what other vars do you mean with “more”? With “ansible -m debug -a “var=hostvars” ansible” I don’t get any other? So lazy loading of the inventory hostvars (in the meaning of “vars of a host in the inventory”) or some other options for limitation isn’t a good idea? esco ---
**Author:** ![Brian\_Coca](https://avatars.discourse-cdn.com/v4/letter/b/6de8d8/32.png) [@Brian\_Coca](https://forum.ansible.com/u/Brian_Coca)\ **Post date:** [September 10, 2015, 9:40pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/16 "2015-09-10T21:40:20Z")
hostvars is a class that contains all vars associated with the host, if you don't add any vars outside of inventory you'll only have the inventory vars there. hostvars itself is lazily loaded, inventory is not. When you query hostvars['myhost'] it will see if it is populated, if not it will ask the inventory object (not script or file) which already loaded and flattened all the vars for all hosts, for the vars in that host to combine with the other sources (play, facts, special vars, etc). As mentioned above, limiting inventory can make a lot of data unavailable that users and plays expect to be there, it should not be done w/o strict control of what runs and knowning that scope well, I would not make this a default option and would hesitate to make it generic as most inventory scripts do not support it. There are some workarounds mentioned above that could be made to work, but i have yet to see a PR that does this cleanly. ---
**Author:** ![Edward\_Zambrano](https://avatars.discourse-cdn.com/v4/letter/e/4da419/32.png) [@Edward\_Zambrano](https://forum.ansible.com/u/Edward_Zambrano)\ **Post date:** [September 17, 2015, 12:07pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/17 "2015-09-17T12:07:26Z")
thank you all for your replies, interesting discussions, One solution we came up that would be backwards compatible is to pass a boolean flag (for now --pass-limit-to-inventory ) when running ansible-playbook that would run the inventory passing the --limit as an argument so. Case 1: my plugin would fail if --limit is pass solution: don’t use --pass-limit-to-inventory Case 2: my plugin expects --limit solution: use --pass-limit-to-inventory when running the ansible-playbook what do you think? ---
**Author:** ![Edward\_Zambrano](https://avatars.discourse-cdn.com/v4/letter/e/4da419/32.png) [@Edward\_Zambrano](https://forum.ansible.com/u/Edward_Zambrano)\ **Post date:** [September 21, 2015, 1:36pm UTC](https://forum.ansible.com/t/pass-limit-to-inventory-to-improve-speed-and-remove-unnecessary-variables/19198/18 "2015-09-21T13:36:34Z")
Hello again, PR done, [https://github.com/ansible/ansible/pull/12451](https://github.com/ansible/ansible/pull/12451), please check it out