Pip module issue - install seems to hang when installing from a requirements file

Archives Ansible Project
1

Can anyone help with this issue…

When log into the remote machine and manually pip install -r production.txt it works fine

In my playbook i have:

  • name: Update the virtualenv with production requirements.
    pip: requirements=${project_root}/myapp/deps/production.txt virtualenv=${project_root}/env extra_args=‘–exists-action w’

Also Tried:

  • name: pip install prod requirements
    command: “${project_root}/env/bin/pip install -r ${project_root}/myapp/deps/production.txt --exists-action w --log ${project_root}/downloads/mypip_log.txt”

ansible 1.2.2
python 2.7.4
pip 1.4.1

remote machine - Ubuntu 13.04

ansible log:

TASK: [Upgrade the virtualenv with prod requirements.] ************************
<198.58.115.201> ESTABLISH CONNECTION FOR USER: root
<198.58.115.201> EXEC [‘sshpass’, ‘-d9’, ‘ssh’, ‘-tt’, ‘-q’, ‘-o’, ‘ControlMaster=auto’, ‘-o’, ‘ControlPersist=60s’, ‘-o’, ‘ControlPath=/tmp/ansible-ssh-%h-%p-%r’, ‘-o’, ‘Port=22’, ‘-o’, ‘GSSAPIAuthentication=no’, ‘-o’, ‘PubkeyAuthentication=no’, ‘-o’, ‘User=root’, ‘-o’, ‘ConnectTimeout=999999’, ‘198.58.115.201’, “/bin/sh -c ‘mkdir -p /tmp/ansible-1376450420.48-23411754031827 && chmod a+rx /tmp/ansible-1376450420.48-23411754031827 && echo /tmp/ansible-1376450420.48-23411754031827’”]
<198.58.115.201> REMOTE_MODULE pip requirements=/home/myuser/projects/myapp/deps/production.txt virtualenv=/home/myuser/projects/env extra_args=‘–exists-action w --log ~/pip_prod1_log.txt’
<198.58.115.201> PUT /tmp/tmpShRoOh TO /tmp/ansible-1376450420.48-23411754031827/pip
<198.58.115.201> EXEC [‘sshpass’, ‘-d9’, ‘ssh’, ‘-tt’, ‘-q’, ‘-o’, ‘ControlMaster=auto’, ‘-o’, ‘ControlPersist=60s’, ‘-o’, ‘ControlPath=/tmp/ansible-ssh-%h-%p-%r’, ‘-o’, ‘Port=22’, ‘-o’, ‘GSSAPIAuthentication=no’, ‘-o’, ‘PubkeyAuthentication=no’, ‘-o’, ‘User=root’, ‘-o’, ‘ConnectTimeout=999999’, ‘198.58.115.201’, “/bin/sh -c ‘chmod a+r /tmp/ansible-1376450420.48-23411754031827/pip’”]
<198.58.115.201> EXEC [‘sshpass’, ‘-d9’, ‘ssh’, ‘-tt’, ‘-q’, ‘-o’, ‘ControlMaster=auto’, ‘-o’, ‘ControlPersist=60s’, ‘-o’, ‘ControlPath=/tmp/ansible-ssh-%h-%p-%r’, ‘-o’, ‘Port=22’, ‘-o’, ‘GSSAPIAuthentication=no’, ‘-o’, ‘PubkeyAuthentication=no’, ‘-o’, ‘User=root’, ‘-o’, ‘ConnectTimeout=999999’, ‘198.58.115.201’, u’/bin/sh -c 'sudo -k && sudo -H -S -p “[sudo via ansible, key=ttlhctamxfmseklxzjnlbwbbtkptumux] password: " -u myuser /bin/sh -c '”'“'/usr/bin/python /tmp/ansible-1376450420.48-23411754031827/pip'”'"''']

pip log:

it looks like the install hangs when it gets here

Downloading/unpacking xhtml2pdf==0.0.3 (from -r /home/myuser/projects/myapp/deps/production.txt (line 15))

Getting page https://pypi.python.org/simple/xhtml2pdf/
URLs to search for versions for xhtml2pdf==0.0.3 (from -r /home/myuser/projects/myapp/deps/production.txt (line 15)):

2

Sounds like this is just a problem with the remote source timing out?

Remember to use {{ foo }} vs $foo BTW :slight_smile:

3

Thank you for the response. I initially thought it was a timeout issue but i have been able to pip install my requirements file multiple times manually on the remote server. I also wrote a python script that parsed my requirements file and pip installed each package individually. I called the script in my playbook and got it to work. But I rather not use the script as a workaround for the pip module.

4

There are lots of people using the pip module, so I don’t know what’s going on on your end.

If you can help debug using hacking/test-module in the checkout that would be useful.

5

i will try that. I have narrowed it down to these two packages in my requirements file…

=1.7.7.1
-e git+https://github.com/staale/python-xlsx@976cb845080aeff5e5a748c9bd28128fc916d5ac#egg=python-xlsx

the pip install does not even attempt to install these packages…as there is no output in the pip log when I try these two individually as the only packages in the requirements file. However, i was able to get my entire requirements file to install successfully including these two packages when i tested it using localhost. It only seems to cause an issue on a remote machine and when they are in a requirements file. I have attempted it on both digital ocean and linode.

Going to keep looking at my environment and try from a different machine.

I appreciate your help

6

I had a similar problem that pip failed to reach pypi.python.org completely when called from ansible. The reason was that a proxy was needed on the remote host and the proxy to connect to was set in the environment variables http_proxy and https_proxy. Since sudo resets the environment for security reasons, the pip command run from ansible with sudo was not able to connect to pypi.python.org.

You can solve this by adding the following line to the sudoers config:
Defaults env_keep += “http_proxy https_proxy”

7

See the “environment” parameter in Ansible for how to pass an arbitrary environment to any task.

http://www.ansibleworks.com/docs/playbooks2.html#setting-the-environment-and-working-with-proxies

This is better because it doesn’t rely on any pre-existing user configuration on the system, or trusting the user’s configuration.