git module profiling

Archives Ansible Project
1

Hello,

I’m trying to find out why the git module is so incredibly slow.

  • I’d like to fix it if possible
  • I need a better profiling tool than eyeballing git module run in htop on the remote host.

I tried https://github.com/rkern/line_profiler but I couldn’t get anywhere, probably because I’m a python newbie.

Would anyone be able give me a few pointers on how to profile ansible module code?

Thanks,

Josef

2

There may be better methods but I'd just instrument the code to spit
out timestamps at various points in its run. I'd use this method
because the git module invokes the git command line frequently so
chances are that one of the git command line calls is what's slow.
Putting some code to log the time around the calls to external git
will let you track down which call it is and also doesn't require you
to learn a lot of python to get started.

For doing this quick and dirty, I would use the python-q library which
does simplistic logging to $TMPDIR/q: https://pypi.python.org/pypi/q
With it you can put lines in your code like:

import q, time; q.q('before: %s' % time.time())
# [git module gets invoked]
q.q('after: %s' % time.time())

Then $TMPDIR/q on the box ansible is talking to will contain these timestamps.

If you want to get more involved, you can look into several modules
from the python stdlib including timeit
https://docs.python.org/2/library/timeit.html and profile:
https://docs.python.org/2/library/profile.html but these will require
learning a bit more python and probably aren't needed for the
granularity of this particular problem.

-Toshio

3

Can you please define slow and explain a bit about how you are testing it?