solved a performance problem in host pattern enumeration

Been running ansible 1.4-devel (a version from september 26, around fd101dd), I yesterday updated to latest (e87be94 22 hours ago 1.3.3 missing from changelog - Michael DeHaan (HEAD, ansible/devel, devel) and noticed a big performance difference.

Runnig ansible 20130926:

PLAY RECAP ********************************************************************
Total elapsed time: 0:00:06.829

with ansible 20131010:

PLAY RECAP ********************************************************************
Total elapsed time: 0:00:41.814

(this is from a playbook where tasks are all skipped, so only inventory and local playbook and runner code involved)

According to git bisect, this is the patch that introduced the change:

8fc46a3a5a286bee5e429b921ef8af9ea7ecbc31 is the first bad commit
commit 8fc46a3a5a286bee5e429b921ef8af9ea7ecbc31
Author: Michael DeHaan <michael@ansibleworks.com>

Return inventory objects in the order they are presented. Additionally, fix host slicing such that it works on Python terms with
zero indexed lists and a non-inclusive final element.

I tracked this down to this method, which mentions problem & solution:

250 # TODO: cache this logic so if called a second time the result is not recalculated
251 def _hosts_in_unenumerated_pattern(self, pattern):

Wrote a patch to implement this caching, and things are now even better than before:

PLAY RECAP ********************************************************************
Total elapsed time: 0:00:03.566

(FYI, my inventory is around 700 hosts in ini files, so these kind of things tend to hit me soon)

PR for my patch: https://github.com/ansible/ansible/pull/4429

Serge

I went ahead and merged this, thanks.

Thinking we need to make sure this doesn’t confuse usage of the “add_host” module.

I am thinking add_host usage should maybe invalidate this cache?

​I'll have a look at https://github.com/ansible/ansible/issues/4442
tonight.​

I’m taking a look at this right now.

I’ll let you know if I can’t resolve it.