# Ansible virtual environments

**URL:** <https://forum.ansible.com/t/ansible-virtual-environments/11254>\
**Category:** Project Discussions\
**Tags:** venv, awx, collections, galaxy-ng, ee, ansible-core\
**Created:** [December 1, 2024, 3:39pm UTC](https://forum.ansible.com/t/ansible-virtual-environments/11254 "2024-12-01T15:39:42Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\
**Post date:** [December 1, 2024, 3:39pm UTC](https://forum.ansible.com/t/ansible-virtual-environments/11254/1 "2024-12-01T15:39:42Z")

</div>

When using Ansible locally, you can either install the `ansible` package (to get ansible-core with a set of collections), or install `ansible-core` and then use `ansible-galaxy collection install` to install some collections. (You can of course also install roles.)

This is a rather common way to use Ansible (at least I assume so), but it has some downsides:

1. If you install `ansible`/`ansible-core` globally, possibly from the system repositories, you might not be able to administer remote systems that have a no longer supported Python version. (See for example [http://forum.ansible.com/t/11198.](http://forum.ansible.com/t/11198.))
2. You have to install needed Python dependencies on the controller yourself. If you’re using system packages for `ansible`/`ansible-core`, you also want to install the packages on system level. But maybe not all packages you need are available.
3. If you need to install additional Python dependencies, you might want to use a Python venv to avoid dependency hell. But telling Ansible to use the venv on the controller side isn’t trivial either - if you ever used `delegate_to: localhost` combined with a venv for the controller, you know how much fun it can be.

One solution to make life simpler is to use execution environments: they encapsulate both problems by using container images, where the version of ansible-core you need, the collections you need, and their Python and system controller dependencies are installed - all based on a rather simple specification.

While execution environments are great when you’re using AWX, they are a bit more cumbersome to use locally:

1. Images are less light-weight than just installing stuff locally.
2. Giving EEs access to parts of the controller - say your SSH keys (potentially coming from hardware tokens), access to cloud/… credentials you have installed locally, etc. - is not trivial.

In the Python world, you’d use a virtual environment instead of a container. So I was wondering: why don’t we have something like virtual environments for Ansible? Basically a Python venv (for ansible-core and the controller Python dependencies) combined with `ANSIBLE_COLLECTIONS_PATH` set to the same directory structure, a way to install collections _together with their Python dependencies_, and some more options set so that Ansible will only use the collections inside the virtual environment, and that `delegate_to: localhost` “just works” (i.e. uses the same venv).

That would make using Ansible locally a lot easier, especially if you need to handle multiple projects using Ansible, each project having its own ansible-core version, its own set of collections (potentially with contradicting version ranges), its own set of controller Python dependencies. Switching could be as simple as switching between Python venvs (for them there are helper programs that automatically activate a venv when you enter the project directory).

What do you think?

---

<div class="post-metadata">

**Author:** ![johan](https://avatars.discourse-cdn.com/v4/letter/j/dec6dc/32.png) [@johan](https://forum.ansible.com/u/johan)\
**Post date:** [December 2, 2024, 9:26am UTC](https://forum.ansible.com/t/ansible-virtual-environments/11254/2 "2024-12-02T09:26:26Z")

</div>

I like the idea, because sometimes you need to be able to use a specific version of a collection.  
That being said, at first, it seems to me it could be solved with a high level script taking care of the creation of a virtual-env, the installation of the desired ansible(core) version, and setting at least an ansible.cfg with the desired ANSIBLE\_COLLECTIONS\_PATH setting.

---

<div class="post-metadata">

**Author:** ![bcoca](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/bcoca/32/9269_2.png) [@bcoca](https://forum.ansible.com/u/bcoca)\
**Post date:** [December 2, 2024, 6:57pm UTC](https://forum.ansible.com/t/ansible-virtual-environments/11254/3 "2024-12-02T18:57:21Z")

</div>

I do this by setting up a Python venv and add `ANSIBLE_COLLECTIONS_PATH` to bin/activate

---

<div class="post-metadata">

**Author:** ![sivel](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sivel/32/1198_2.png) [@sivel](https://forum.ansible.com/u/sivel)\
**Post date:** [December 2, 2024, 7:50pm UTC](https://forum.ansible.com/t/ansible-virtual-environments/11254/4 "2024-12-02T19:50:17Z")

</div>

virtual environments aren’t really portable. But something like `shiv` could offer something similar:

```auto
$ shiv -o ansible.pyz -e ansible. __main__ :main ansible-core
[...]
$ ./ansible.pyz adhoc --version
ansible [core 2.18.1]
  config file = None
  configured module search path = ['/Users/sivel/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/sivel/.shiv/ansible.pyz_87c30e3479034dddc40497d3460c9686499dd0bab0d1b39b45e227c989a22848/site-packages/ansible
  ansible collection location = /Users/sivel/.ansible/collections:/usr/share/ansible/collections
  executable location = ./ansible.pyz
  python version = 3.11.0 (main, Nov 17 2022, 20:05:40) [Clang 14.0.0 (clang-1400.0.29.102)] (/Users/sivel/venvs/shiv/bin/python3)
  jinja version = 3.1.4
  libyaml = True

```

---

<div class="post-metadata">

**Author:** ![oranod](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/oranod/32/1434_2.png) [@oranod](https://forum.ansible.com/u/oranod)\
**Post date:** [December 3, 2024, 8:02pm UTC](https://forum.ansible.com/t/ansible-virtual-environments/11254/5 "2024-12-03T20:02:24Z")

</div>

@felixfontein I’d recommend looking at Ansible dev environments: [Ansible Development Environment Documentation](https://ansible.readthedocs.io/projects/dev-environment/#features)

I know there are lots of options out there but I believe this project is specifically designed for the use case you’ve put forward. I’d definitely encourage you to have a go with it and open some issues (or send PRs) if you see ways to improve it.

---

<div class="post-metadata">

**Author:** ![cidrblock](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/cidrblock/32/126_2.png) [@cidrblock](https://forum.ansible.com/u/cidrblock)\
**Post date:** [December 3, 2024, 9:06pm UTC](https://forum.ansible.com/t/ansible-virtual-environments/11254/6 "2024-12-03T21:06:56Z")

</div>

Thanks Don.

Yeah, this is exactly what we were trying to solve with ansible-dev-environment.

It’ll build a venv, installs all the dev tools, install ansible-core as a result, installs collections into the python venv where ansible can find them, removes collections, install python deps for collections using the dep resolver from builder and some other fun CLI stuff.

My favorite feature is the ability to install a collection as “editable”, just like pip can so dev is much smoother.

Have a look, PRs, feedback, etc all welcome. It’s a fairly new project so there might be some wrinkles there still 🙂

---

<div class="post-metadata">

**Author:** ![Jeff\_Pullen](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/jeff_pullen/32/5318_2.png) [@Jeff\_Pullen](https://forum.ansible.com/u/Jeff_Pullen)\
**Post date:** [December 3, 2024, 9:35pm UTC](https://forum.ansible.com/t/ansible-virtual-environments/11254/7 "2024-12-03T21:35:28Z")

</div>

Something about python virtual environments has never worked well for me. At least not having to switch between different ones. It would be very helpful if there was an option in the ansible.cfg to just enable automatic venv configuration and creation.

I’ve almost exclusively switched over to using the Devfiles and Devcontainers these days. It suits my work/lifestyle of jumping in and out of different projects and between different computers without leaving a cluttered mess behind me. Plus I don’t have to remember where I created that venv or to actually activate it any more.

---

<div class="post-metadata">

**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\
**Post date:** [February 5, 2025, 7:34am UTC](https://forum.ansible.com/t/ansible-virtual-environments/11254/8 "2025-02-05T07:34:30Z")

</div>

Sorry for taking so long, I finally got around to try out ansible-dev-environment.

When trying to use `ade install`, I get:

```auto
   Error: Collections found in /home/felix/.ansible/collections/ansible_collections
    Hint: Run `rm -rf ~/.ansible/collections` to remove them.
Critical: The development environment is not isolated, please resolve the above errors.

```

After deleting `~/.ansible/collections` (in my case it only contained a symlink for some experiment that no longer worked from some years ago), it kind of worked, _but_ since it doesn’t isolate the virtual environment, all the collections I have on `ANSIBLE_COLLECTIONS_PATH` bleed into this virtual environment.

Also it did not install the Python dependencies of the collections. (From the EE Python requirements.)

Also I’m not sure how to select the ansible-core version. It seems to be installed into the venv as part of the `ansible-dev-environment` package, and it simply seems to be the latest release version.

So at least for the use-case I described in the original post, having **isolated** ansible environments for specific ansible-core versions with a specific set of collections and their Python requirements (basically an EE without a container), this doesn’t seem to work well at the moment.

---

<div class="post-metadata">

**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\
**Post date:** [February 5, 2025, 10:23pm UTC](https://forum.ansible.com/t/ansible-virtual-environments/11254/9 "2025-02-05T22:23:27Z")

</div>

I created two issues for that:

- [RFE: Isolate environments by setting ANSIBLE\_COLLECTIONS\_PATH · Issue #294 · ansible/ansible-dev-environment · GitHub](https://github.com/ansible/ansible-dev-environment/issues/294)
- [RFE: Allow to specify ansible-core version on `ade install` · Issue #293 · ansible/ansible-dev-environment · GitHub](https://github.com/ansible/ansible-dev-environment/issues/293)