How to connect to test instances with dynamic ips?

Get Help
, , ,
1

I am developing an ansible collection.

I am reading about and working with the Ansible-Native configuration and want to use the shared state feature (Ansible-Native Configuration - Ansible Molecule).

However it seems like molecule can not find instances with dynamic ip addresses. I populated and persistet the instance_config.yml:

- address: 192.168.123.64
  become_method: sudo
  identity_file: /home/chronicc/.ansible/tmp/molecule.kBwO.default/id_ed25519
  instance: instance01
  port: 22
  user: molecule

I’ve tinkered around with the config.yml and are now at a point where the shared state setup does block itself.

I am currently relying on the inventory file that is created by molecule itself (MOLECULE_INVENTORY_FILE ).

❯ cat /home/chronicc/.ansible/tmp/molecule.kBwO.default/inventory/ansible_inventory.yml 
# Molecule managed

---
all:
  hosts:
    instance01: &id001
      ansible_become_method: sudo
      ansible_connection: smart
      ansible_host: 192.168.123.64
      ansible_port: 22
      ansible_private_key_file: /home/chronicc/.ansible/tmp/molecule.kBwO.default/id_ed25519
      ansible_ssh_common_args: -o UserKnownHostsFile=/dev/null -o ControlMaster=auto
        -o ControlPersist=60s -o ForwardX11=no -o LogLevel=ERROR -o IdentitiesOnly=yes
        -o StrictHostKeyChecking=no
      ansible_user: molecule
  vars: &id002
    molecule_ephemeral_directory: '{{ lookup(''env'', ''MOLECULE_EPHEMERAL_DIRECTORY'')
      }}'
    molecule_file: '{{ lookup(''env'', ''MOLECULE_FILE'') }}'
    molecule_instance_config: '{{ lookup(''env'', ''MOLECULE_INSTANCE_CONFIG'') }}'
    molecule_no_log: '{{ lookup(''env'', ''MOLECULE_NO_LOG'') or not molecule_yml.provisioner.log|default(False)
      | bool }}'
    molecule_scenario_directory: '{{ lookup(''env'', ''MOLECULE_SCENARIO_DIRECTORY'')
      }}'
    molecule_yml: '{{ lookup(''file'', molecule_file) | from_yaml }}'
kubernetes_control_nodes:
  hosts:
    instance01: *id001
  vars: *id002
ungrouped:
  vars: {}

However when I am running my scenario kubernetes, it want’s to use the ansible_inventory.yml from it’s own ephemeral directory which does not contain the populated inventory file listed above.

Before this, I used the inventory directory extensions/molecule/inventory. This is accessible from all scenarios so the shared state feature does work. However this inventory does not get updated with the dynamic ip addresses for my instances. I am using libvirt currently but this can also change to vSphere or AWS.

I had hoped that molecule somehow combines the static inventory with the test instances hostnames and variables with the dynamic inventory from the ephemeral directory. Or somehow reuses the ip addresses from the instance_config.yml to update the static inventory in memory before running the playbooks.

So how would I need to setup Ansible-Native configuration with shared state and dynamic ip addresses?

2

As far as I understand, when using Ansible-Native configuration, you are on your own. Molecule will not do any magic for you. This means:

  1. You will have to generate your inventory and specify Ansible to use it in molecule configuration:
  executor:
    backend: ansible-playbook
    args:
      ansible_playbook:
        - --inventory=/path/to/my_inv.yml

  1. For create phase you will implement a playbook that does all the provisioning of instances and/or containers and generate my_inv.yml.

  2. Other phases will read the generated inventory to run the test against.

In theory, your my_inv.yml can be a static file by using appropriate inventory plugin to dynamically inspect parameters of your newly provisioned instances. This can work for AWS for example:

  1. Your create.yml playbook provisions the instances using ec2_instance module. No additional work needed.
  2. Once your converge.yml playbook runs, Ansible will use inventory that uses aws_ec2 inventory plugin to dynamically retrieve instance information. There is no need to persist, transfer or share any inventory related info between these phases.
3

Hello @bvitnik and thank you for your answer.

I believe that I did not do well to describe my cause. My problem is not to share an inventory between the phases (e.g. create, converge, destroy) but between scenarios (e.g. default, kubernetes, etc.).

Multiple scenarios will create multiple ephemeral directories in the ansible temporary directory:

❯  ls -l .ansible/tmp/
total 40
...
drwxrwxr-x 2 chronicc chronicc 4096 Mar 28 23:50 molecule.kBwO
drwxrwxr-x 4 chronicc chronicc 4096 Apr  5 20:34 molecule.kBwO.default
drwxrwxr-x 3 chronicc chronicc 4096 Apr  5 20:13 molecule.kBwO.kubernetes

There you can see the first directory is holding a state.yml file which tells molecule that the “resource stack” (e.g. the virtual machines) have been created.

The second directory is tracking the inventory from the default scenario which is used to create and destroy the virtual machines. This scenario has only create and destroy stage like described in the documentation.

The third directory is used to run the actual converge stage from the kubernetes scenario.

What works

I can create virtual machines with libvirt. They have dynamic ip addresses and I can do molecule login. I can also destroy those machines. The ephemeral directory also contains the generated inventory with all necessary variables.

What doesn work

The ephemeral directory of the kubernetes scenario does contain another inventory. This inventory does not have the variables required for connecting to the virtual machines (e.g. ip addresses, ansible_user).

My question

How can the kubernetes scenario reliable connect to the virtual machines to run the converge phase. Or how can the kubernetes scenario reuse the inventory of the default scenario.

Regarding the documentation of molecule this should be possible somehow. I fear it’s not explained in a way I can understand how.

4

Can you share your base config.yml and per scenario molecule.yml files?

I think that what I described still holds. Something tells me that you are maybe mixing traditional and Ansible-Native approach.

Can you also share your default scenario create.yml?

That would clarify your situation.

5

Yes, I am mixing both the old and the new way. If this is not recommended how would I then spawn test resources? In my understanding the whole point of molecule is to spawn test resources and run automated playbooks.

My Configs:

# config.yml
---
ansible:
  cfg:
    defaults:
      stdout_callback: default
    inventory:
      enable_plugins: yaml
  executor:
    backend: ansible-playbook
    args:
      ansible_playbook:
        - --inventory=$MOLECULE_INVENTORY_FILE

provisioner:
  name: ansible
  env:
    ANSIBLE_VERBOSITY: 4
  inventory:
    group_vars:
      all:
        collection_namespace: chronicc
        collection_name: toolchain
        molecule_image_registry: http://worker02.neb.kurthos.site/virtualization/images/
  options:
    diff: false

role_name_check: 1

scenario:
  test_sequence:
    - prepare
    - converge
    - verify
    - idempotence
    - verify
    - cleanup

shared_state: true

verifier:
  name: ansible


# default molecule.yml
---
platforms:
  - name: instance01
    cpus: 1
    groups:
      - kubernetes_control_nodes
    image: noble-server-cloudimg-amd64.img
    memory: 2048
    ostype: ubuntu24.04

scenario:
  test_sequence:
    - create
    - destroy

The kubernetes scenario molecule.yml is empty.

As far as I understood, the “old way” with provisioners is deprecated and will be removed in future versions. Hence the benefit of molecule would be gone if provisioning is not a part of it anymore. Not blaming here, just want to know if I understand this correctly.

6

I don’t see your driver section so I don’t know how you configured molecule to use libvirt.

Anyway, you cannot mix traditional and Ansible-Native approach. As soon as molecule sees platforms section in your config, it will assume you are using traditional approach.

I think that your example is one of those that showed the weakness of traditional Molecule. Many things, including how drivers worked, were hard coded and could not be workarounded.

So in Ansible-Native approach, instead of using a driver (libvirt in your example) you effectively implement your own by making a create.yml playbook that provisions your test instances. The molecule drivers were Ansible playbooks anyway but with Ansible-Native approach you now have a complete control of the provisioning process.

So what I initially said still holds. With Ansible-Native approach you have to:

  1. Implement your own provisioning process in create.yml playbook. In your case you would use community.libvirt.virt module to get your instances up and running. You can use the create.yml playbook from the existing libvirt molecule driver as an inspiration:

Just a warning that it is overly complex because it tries to cover a lot of different environments and requirements. You can probably make it much simpler.

  1. In the same create.yml playbook, implement a few tasks that generate your inventory (write a file) at some location that is accessible and referenced by all scenarios. Alternatively, you can use community.libvirt.libvirt dynamic inventory plugin to dynamically discover your instances.

If you want to stick to the traditional way, try creating per scenario molecule.yml that contains the following section:

ansible:
  executor:
    backend: ansible-playbook
    args:
      ansible_playbook:
        - --inventory=/path/to/the/inventory/created/by/default/scenario

UPDATE:

I think this documentation page illustrates Ansible-Native approach of inventory handling well:

It uses containers as test instances but the only thing you have to change is what module is used for instance provisioning, like community.libvirt.virt instead of containers.podman.podman_container as shown in the example. Of course, there is additional stuff to do but you can use it as a skeleton of your create.yml playbook.

7

I did not configure the driver since I don’t want to use the original libvirt driver. So it probably is default. I used the new and old way both in a project inside my company and it works without problems. Only difference here is the shared_state option which is true for this project.

The code for the project described in this post is here: GitHub - chronicc/ansible-collection-toolchain at kubernetes-role · GitHub

I could’ve shared this earlier now that I post it.

Okay what I understood is that I need a to create a second inventory inside my create playbook which then will be used by the other scenarios. I was unsure if this is the way reading the documentation.

I will go this route and implement it to see how this actually works. Thanks for your input so far.
Will report back when I have closure.

8

Found this known issue that seems to be related:

So it seems that shared_state does not work correctly in general as inventory is not shared. This is unlikely to be fixed because this is specific to traditional approach. With Ansible-Native approach you must handle this yourself anyway.

Now that I see your create.yml it now makes sense how you ended up in a situation where you don’t use any driver (defaults to delegated) but have instance_config.yml created which is specific to driver usage. Your create.yml explicitly creates it (last two tasks). It looks to me as if they were copy-pasted from the libvirt driver (AI generated?). instance_config.yml is kinda part of the protocol of communication between molecule and a driver but is meaningless with Ansible-Native approach. You make your own protocol including how inventory is shared between scenarios.

I personally use shared_state for testing but with systemd enabled Docker images so things are much simpler and no fiddling with inventory is necessary.

9

Hello,

yes I reused some code from the libvirt plugin.

I’ve solved my problem with the unshared inventory. Instead of the instance_config.yml I now write a second inventory file into extensions/molecule/inventory/ which is generated from the created instances. This forces me to add a .gitignore entry for the generated inventory but solves the problem of the same inventory for different scenarios for now.

The ticket you mentioned was the reason I registered with this forum since there are no answers on the ticket that helped me. :slight_smile: I’d like this issue to be solved so that I can remove my workaround.

Unfortunately without the platform key in molecule.yml I can not longer use molecule login since the method does not read the inventory and thus is not supported with the Ansible-Native way. However this is a small price to pay for the freedom of having a working libvirt setup that I can maintain easily.

I’ve pushed the code to my above mentioned repository if you want to take a look. Probably gonna write a blog post in the next few days to document my findings. Again, thanks for your help.

10

Glad to be of help :smiley:. Cheers.