Ansible virtual environments

Project Discussions
, , , , ,
1

When using Ansible locally, you can either install the ansible package (to get ansible-core with a set of collections), or install ansible-core and then use ansible-galaxy collection install to install some collections. (You can of course also install roles.)

This is a rather common way to use Ansible (at least I assume so), but it has some downsides:

  1. If you install ansible/ansible-core globally, possibly from the system repositories, you might not be able to administer remote systems that have a no longer supported Python version. (See for example http://forum.ansible.com/t/11198.)
  2. You have to install needed Python dependencies on the controller yourself. If you’re using system packages for ansible/ansible-core, you also want to install the packages on system level. But maybe not all packages you need are available.
  3. If you need to install additional Python dependencies, you might want to use a Python venv to avoid dependency hell. But telling Ansible to use the venv on the controller side isn’t trivial either - if you ever used delegate_to: localhost combined with a venv for the controller, you know how much fun it can be.

One solution to make life simpler is to use execution environments: they encapsulate both problems by using container images, where the version of ansible-core you need, the collections you need, and their Python and system controller dependencies are installed - all based on a rather simple specification.

While execution environments are great when you’re using AWX, they are a bit more cumbersome to use locally:

  1. Images are less light-weight than just installing stuff locally.
  2. Giving EEs access to parts of the controller - say your SSH keys (potentially coming from hardware tokens), access to cloud/… credentials you have installed locally, etc. - is not trivial.

In the Python world, you’d use a virtual environment instead of a container. So I was wondering: why don’t we have something like virtual environments for Ansible? Basically a Python venv (for ansible-core and the controller Python dependencies) combined with ANSIBLE_COLLECTIONS_PATH set to the same directory structure, a way to install collections together with their Python dependencies, and some more options set so that Ansible will only use the collections inside the virtual environment, and that delegate_to: localhost “just works” (i.e. uses the same venv).

That would make using Ansible locally a lot easier, especially if you need to handle multiple projects using Ansible, each project having its own ansible-core version, its own set of collections (potentially with contradicting version ranges), its own set of controller Python dependencies. Switching could be as simple as switching between Python venvs (for them there are helper programs that automatically activate a venv when you enter the project directory).

What do you think?

2 Likes
2

I like the idea, because sometimes you need to be able to use a specific version of a collection.
That being said, at first, it seems to me it could be solved with a high level script taking care of the creation of a virtual-env, the installation of the desired ansible(core) version, and setting at least an ansible.cfg with the desired ANSIBLE_COLLECTIONS_PATH setting.

3

I do this by setting up a Python venv and add ANSIBLE_COLLECTIONS_PATH to bin/activate

4

virtual environments aren’t really portable. But something like shiv could offer something similar:

$ shiv -o ansible.pyz -e ansible.__main__:main ansible-core
[...]
$ ./ansible.pyz adhoc --version
ansible [core 2.18.1]
  config file = None
  configured module search path = ['/Users/sivel/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/sivel/.shiv/ansible.pyz_87c30e3479034dddc40497d3460c9686499dd0bab0d1b39b45e227c989a22848/site-packages/ansible
  ansible collection location = /Users/sivel/.ansible/collections:/usr/share/ansible/collections
  executable location = ./ansible.pyz
  python version = 3.11.0 (main, Nov 17 2022, 20:05:40) [Clang 14.0.0 (clang-1400.0.29.102)] (/Users/sivel/venvs/shiv/bin/python3)
  jinja version = 3.1.4
  libyaml = True
1 Like
5

@felixfontein I’d recommend looking at Ansible dev environments: Ansible Development Environment Documentation

I know there are lots of options out there but I believe this project is specifically designed for the use case you’ve put forward. I’d definitely encourage you to have a go with it and open some issues (or send PRs) if you see ways to improve it.

1 Like
6

Thanks Don.

Yeah, this is exactly what we were trying to solve with ansible-dev-environment.

It’ll build a venv, installs all the dev tools, install ansible-core as a result, installs collections into the python venv where ansible can find them, removes collections, install python deps for collections using the dep resolver from builder and some other fun CLI stuff.

My favorite feature is the ability to install a collection as “editable”, just like pip can so dev is much smoother.

Have a look, PRs, feedback, etc all welcome. It’s a fairly new project so there might be some wrinkles there still :slight_smile:

1 Like
7

Something about python virtual environments has never worked well for me. At least not having to switch between different ones. It would be very helpful if there was an option in the ansible.cfg to just enable automatic venv configuration and creation.

I’ve almost exclusively switched over to using the Devfiles and Devcontainers these days. It suits my work/lifestyle of jumping in and out of different projects and between different computers without leaving a cluttered mess behind me. Plus I don’t have to remember where I created that venv or to actually activate it any more.

1 Like