Is it possible to upgrade libraries in awx?

Hi,

I want to upgrade the following libraries in awx containers (possibly during build process). Could anyone please suggest a way to achieve this. Also, since awx supports only Python3, can anyone please suggest why we have python 2.x available in the task and web images?

`

/usr/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py
/usr/lib/python2.7/site-packages/pip/_vendor/urllib3/connectionpool.py
/usr/lib/python2.7/site-packages/pip/_vendor/urllib3/poolmanager.py
/usr/lib/python2.7/site-packages/pip/_vendor/urllib3/util/retry.py
/usr/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/connectionpool.py
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/poolmanager.py
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/util/retry.py
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/poolmanager.py
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/util/retry.py
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/botocore/vendored/requests/sessions.py
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/pip/_vendor/requests/packages/urllib3/connectionpool.py
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/pip/_vendor/requests/packages/urllib3/poolmanager.py
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/pip/_vendor/requests/packages/urllib3/util/retry.py
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/pip/_vendor/urllib3/connectionpool.py
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/pip/_vendor/urllib3/poolmanager.py
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/pip/_vendor/urllib3/util/retry.py
/var/lib/awx/venv/awx/lib/python3.6/site-packages/pip/_vendor/requests/packages/urllib3/connectionpool.py
/var/lib/awx/venv/awx/lib/python3.6/site-packages/pip/_vendor/requests/packages/urllib3/poolmanager.py
/var/lib/awx/venv/awx/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py
/var/lib/awx/venv/awx/lib/python3.6/site-packages/pip/_vendor/requests/packages/urllib3/util/retry.py
/var/lib/awx/venv/awx/lib/python3.6/site-packages/pip/_vendor/urllib3/connectionpool.py
/var/lib/awx/venv/awx/lib/python3.6/site-packages/pip/_vendor/urllib3/poolmanager.py
/var/lib/awx/venv/awx/lib/python3.6/site-packages/pip/_vendor/urllib3/util/retry.py

libcomps
libseccomp
sudo
libarchive

`

First on your list, the _vendor modules in the system pip are upgraded by upgrading pip. It would probably be literally pip install --upgrade pip and same for pip3. You could do that in your custom images, sure.

In the build process, the pip that is installed to the awx and ansible venv locations comes from versions specified in requirements/requirements.txt and requirements/requirements_ansible.txt. So you would toggle these versions to update pip in those 2 venvs, as it builds.

I’m working to upgrade these in an open PR for the venv pip versions, but not the system packages.

Hello Alan,

First of all, thanks a lot for your response. Your suggestion worked perfectly. I was able to re-build the images by updating the pip version to latest in requirements.txt and requirements_ansible.txt However, i can still see the following with older versions. Could you please suggest how can i fix the following:

/usr/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py | FIX_VERSION: 2.20.0
/usr/lib/python2.7/site-packages/pip/_vendor/urllib3/connectionpool.py | FIX_VERSION: 1.23
/usr/lib/python2.7/site-packages/pip/_vendor/urllib3/poolmanager.py | FIX_VERSION: 1.23
/usr/lib/python2.7/site-packages/pip/_vendor/urllib3/util/retry.py | FIX_VERSION: 1.23
/usr/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py | FIX_VERSION: 2.20.0
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/connectionpool.py | FIX_VERSION: 1.23
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/poolmanager.py | FIX_VERSION: 1.23
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/util/retry.py | FIX_VERSION: 1.23
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py | FIX_VERSION: 1.23
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/poolmanager.py | FIX_VERSION: 1.23
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/util/retry.py | FIX_VERSION: 1.23
/var/lib/awx/venv/ansible/lib/python3.6/site-packages/botocore/vendored/requests/sessions.py | FIX_VERSION: 2.20.0
libcomps | FIX_VERSION: 0.1.11-2.el8
libseccomp | FIX_VERSION: 2.4.1-1.el8
sudo | FIX_VERSION: 1.8.25p1-8.el8_1
libarchive | FIX_VERSION: 3.3.2-7.el8

Thanks in advance.

botocore removed the requests library from their source in version 1.13.0 (version number of botocore), which can be upgraded in requirements/requirements_ansible.txt, but boto3 specifies a narrow version range for this dependency so really you probably need to upgrade both. Or upgrade boto3 and follow the updater.sh script instructions in the requirements folder. That was already on my agenda, here is a PR for it https://github.com/ansible/awx/pull/5547 (but this is probably more than what you need for now)

For the system pip, vendored requests was upgraded to 2.22.0 in pip version 19.2. I believe this is pip vendored with python. So you could do a yum update of the rpm for this particular python, or something along those lines.

Alan
github: AlanCoding

Hello Alan,

Thanks again, with your suggestions i was able to upgrade all but the following packages:

/usr/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/connectionpool.py
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/poolmanager.py
/usr/lib/python3.6/site-packages/pip/_vendor/urllib3/util/retry.py

Could you please suggest how can i update these. I have checked and pip/pip2/pip3 is upgraded in awx, ansible venv and in default python environment however, these packages are still left. Could you please suggest how can i fix these.

I understand the PR you are working on would be the best approach to upgrade the dependencies since it includes all the test and i will be keeping a check on its update. However, for the current workaround, i need to update these packages to avoid blocking our images in our environment.

Thanks.