# slow inventory build

**URL:** <https://forum.ansible.com/t/slow-inventory-build/37505>\
**Category:** Ansible Project\
**Created:** [December 28, 2022, 4:42pm UTC](https://forum.ansible.com/t/slow-inventory-build/37505 "2022-12-28T16:42:39Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![mdidomenico](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@mdidomenico](https://forum.ansible.com/u/mdidomenico)\
**Post date:** [December 28, 2022, 4:42pm UTC](https://forum.ansible.com/t/slow-inventory-build/37505/1 "2022-12-28T16:42:39Z")

</div>

i have a flat list of hosts, no groups, no variables, total is over  
15k. when i run

$ head inv-expanded.yml  
all:  
&nbsp;&nbsp;hosts:  
&nbsp;&nbsp;&nbsp;&nbsp;admin:  
&nbsp;&nbsp;&nbsp;&nbsp;admin1:  
&nbsp;&nbsp;&nbsp;&nbsp;admin2:  
&nbsp;&nbsp;&nbsp;&nbsp;admin3:  
&nbsp;&nbsp;&nbsp;&nbsp;admin4:  
&nbsp;&nbsp;&nbsp;&nbsp;admin5:  
&nbsp;&nbsp;&nbsp;&nbsp;admin6:  
...snipped...

$ time ansible-inventory -i inv-expanded.yml --list \> a  
real 0m28.334s  
user 0m23.725s  
sys 0m4.452s

$ ansible-inventory --version  
ansible-inventory 2.9.27  
&nbsp;&nbsp;config file = /etc/ansible/ansible.cfg  
&nbsp;&nbsp;configured module search path = ['/usr/share/ansible/plugins/modules']  
&nbsp;&nbsp;ansible python module location = /usr/lib/python3.6/site-packages/ansible  
&nbsp;&nbsp;executable location = /usr/bin/ansible-inventory  
&nbsp;&nbsp;python version = 3.6.8 (default, Oct 20 2022, 09:31:56) [GCC 8.5.0  
20210514 (Red Hat 8.5.0-15)]

$ head -n20 /proc/cpuinfo  
processor : 0  
vendor\_id : AuthenticAMD  
cpu family : 23  
model : 49  
model name : AMD EPYC 7302P 16-Core Processor  
stepping : 0  
microcode : 0x8301055  
cpu MHz : 3000.000  
cache size : 512 KB

when i include the groups and variables this slows down even more.

this ticket [https://github.com/ansible/ansible/issues/30534](https://github.com/ansible/ansible/issues/30534) seems to  
claim that there used to be an issue in the v2.3/v2.4 era, but it was  
fixed and i should be seeing ~6s or less

is there a way to speed this up without splitting the inventory file  
into multiple smaller files?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/ansible/original/1X/1882d9a01da5d8996a862359b0913cc568a39b84.svg) [@system](https://forum.ansible.com/u/system)\
**Post date:** [December 31, 2022, 11:48pm UTC](https://forum.ansible.com/t/slow-inventory-build/37505/2 "2022-12-31T23:48:01Z")

</div>

Do you have libyaml installed? it is the C version of the YAML  
tokenizer, if not the processing will be done in python, which is much  
slower. The difference is not noticeable with small files, but it  
really becomes visible with large ones.

---

<div class="post-metadata">

**Author:** ![mdidomenico](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@mdidomenico](https://forum.ansible.com/u/mdidomenico)\
**Post date:** [January 3, 2023, 3:35pm UTC](https://forum.ansible.com/t/slow-inventory-build/37505/3 "2023-01-03T15:35:59Z")

</div>

it is installed. is there a way for me to confirm that ansible is  
actually using it?

$ rpm -qa | grep -i yaml  
perl-CPAN-Meta-YAML-0.018-397.el8.noarch  
python3-pyyaml-3.12-12.el8.x86\_64  
libyaml-0.1.7-5.el8.x86\_64

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/ansible/original/1X/1882d9a01da5d8996a862359b0913cc568a39b84.svg) [@system](https://forum.ansible.com/u/system)\
**Post date:** [January 5, 2023, 4:52pm UTC](https://forum.ansible.com/t/slow-inventory-build/37505/4 "2023-01-05T16:52:13Z")

</div>

It could be related to this  
[https://github.com/ansible/ansible/issues/79652](https://github.com/ansible/ansible/issues/79652) (see linked PR for  
fix), but you really don't provide enough information.

---

<div class="post-metadata">

**Author:** ![Will\_McDonald](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Will\_McDonald](https://forum.ansible.com/u/Will_McDonald)\
**Post date:** [January 5, 2023, 5:14pm UTC](https://forum.ansible.com/t/slow-inventory-build/37505/5 "2023-01-05T17:14:55Z")

</div>

You can strace the ansible-inventory execution to check what libraries and syscalls are used:

$ strace ansible-inventory -i inventory/hosts --list 2\>&1 | grep libya  
openat(AT\_FDCWD, “/lib64/libyaml-0.so.2”, O\_RDONLY|O\_CLOEXEC) = 3

$ rpm -qf /lib64/libyaml-0.so.2  
libyaml-0.2.5-8.fc37.x86\_64

---

<div class="post-metadata">

**Author:** ![mdidomenico](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@mdidomenico](https://forum.ansible.com/u/mdidomenico)\
**Post date:** [January 5, 2023, 6:48pm UTC](https://forum.ansible.com/t/slow-inventory-build/37505/6 "2023-01-05T18:48:49Z")

</div>

thanks i did see that yesterday when i opened my ticket.  
[https://github.com/ansible/ansible/issues/79664](https://github.com/ansible/ansible/issues/79664)

if there's more info i can provide please let me know

---

<div class="post-metadata">

**Author:** ![mdidomenico](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@mdidomenico](https://forum.ansible.com/u/mdidomenico)\
**Post date:** [January 5, 2023, 6:55pm UTC](https://forum.ansible.com/t/slow-inventory-build/37505/7 "2023-01-05T18:55:12Z")

</div>

ah, yes your right for some reason that didn't jump into my head. i  
am in fact using libyaml

inventory-testing]$ strace ansible-inventory -i ./inv.py --list 1\>a 2\>&1  
[inventory-testing]$ grep libyaml a  
openat(AT\_FDCWD, "/lib64/libyaml-0.so.2", O\_RDONLY|O\_CLOEXEC) = 3

i also found this interesting, seems excessive

[[inventory-testing]$ grep open a | sort | uniq -c | grep -v "^ [1-9]"  
&nbsp;&nbsp;18591 openat(AT\_FDCWD, "/home/user/.ansible/plugins/connection",  
O\_RDONLY|O\_NONBLOCK|O\_CLOEXEC|O\_DIRECTORY) = -1 ENOENT (No such file  
or directory)  
111536 openat(AT\_FDCWD, "/home/user/.ansible/plugins/vars",  
O\_RDONLY|O\_NONBLOCK|O\_CLOEXEC|O\_DIRECTORY) = -1 ENOENT (No such file  
or directory)  
&nbsp;&nbsp;18590 openat(AT\_FDCWD,  
"/usr/lib/python3.6/site-packages/ansible/plugins/connection",  
O\_RDONLY|O\_NONBLOCK|O\_CLOEXEC|O\_DIRECTORY) = 3  
&nbsp;&nbsp;18590 openat(AT\_FDCWD,  
"/usr/lib/python3.6/site-packages/ansible/plugins/connection/\_\_pycache\_\_",  
O\_RDONLY|O\_NONBLOCK|O\_CLOEXEC|O\_DIRECTORY) = 3  
111535 openat(AT\_FDCWD,  
"/usr/lib/python3.6/site-packages/ansible/plugins/vars",  
O\_RDONLY|O\_NONBLOCK|O\_CLOEXEC|O\_DIRECTORY) = 3  
111535 openat(AT\_FDCWD,  
"/usr/lib/python3.6/site-packages/ansible/plugins/vars/\_\_pycache\_\_",  
O\_RDONLY|O\_NONBLOCK|O\_CLOEXEC|O\_DIRECTORY) = 3  
&nbsp;&nbsp;18591 openat(AT\_FDCWD, "/usr/share/ansible/plugins/connection",  
O\_RDONLY|O\_NONBLOCK|O\_CLOEXEC|O\_DIRECTORY) = 3  
111536 openat(AT\_FDCWD, "/usr/share/ansible/plugins/vars",  
O\_RDONLY|O\_NONBLOCK|O\_CLOEXEC|O\_DIRECTORY) = 3  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;22 stat("/usr/lib64/python3.6/site-packages/cryptography/hazmat/backends/openssl",  
{st\_mode=S\_IFDIR|0755, st\_size=4096, ...}) = 0
