Redirecting (type: connection) when using ansible copy during build stage on Azure DevOps build server

Get Help
1

We are using a buildserver and agent to build a application within a conatiner (docker)
Recently we swicthed between Azure projects and altough we seem to have the same configuration for the build servers, we are getting a lot of lagging in building.
Espacially with a copy step.

We also get a message stating:
redirecting (type: connection) ansible.builtin.podman to containers.podman.podman

We use the following copy command:
- name: “Copy cucs.”
copy:
src: “{{ playbook_dir }}/conf/prj_server/rsx”
dest: “{{ appl_path }}/prj_server”
owner: “{{ user }}”
group: “{{ group }}”

This copies the files from our repository to the build server container
In the old situation it took like 10 seconds but now it takes more then 9 minutes.

The copy is abount 350 files
We also thought about synchronize but that does not work with a docker/podman container

2

This is a known issue:

  • The ansible.builtin.copy module recursively copy facility does not scale to lots (>hundreds) of files.

Could you use ansible.builtin.command and scp? for example something like this?

- name: Copy cucs.
  ansible.builtin.command: >-
    scp {{ playbook_dir }}/conf/prj_server/rsx/* \
    {{ inventory_hostname}}:{{ appl_path }}/prj_server/
  delegate_to: localhost
2 Likes
3

The reason why containers.podman.podman (and also community.docker.docker and community.docker.docker_api and other similar connection plugins) are so much slower is that the APIs they use have quite an overhead when executing a command. So if you run quite many commands, which happens in the back when using copy for a large number of files (as @chris pointed out), you will notice an extreme slowdown.

(The SSH connection plugin re-uses the same SSH session, so it’s a lot faster.)

I’m not sure whether the podman collection offers some wrapper for podman cp that allows to copy a whole directory, but if not, you can always use ansible.builtin.command with podman cp, similar to what @chris suggested.

3 Likes
4

It now looks like it is switching on more items.
Almost as that it would not take the builtin modules but the counterparts.

redirecting (type: connection) ansible.builtin.podman to containers.podman.podman

redirecting (type: modules) ansible.builtin.authorized_key to ansible.posix.authorized_key

redirecting (type: modules) ansible.builtin.podman_image_info to containers.podman.podman_image_info

redirecting (type: callback) ansible.builtin.yaml to community.general.yaml

redirecting (type: callback) ansible.builtin.yaml to community.general.yaml

redirecting (type: callback) ansible.builtin.profile_roles to ansible.posix.profile_roles

redirecting (type: callback) ansible.builtin.profile_tasks to ansible.posix.profile_tasks

redirecting (type: modules) ansible.builtin.podman_image_info to containers.podman.podman_image_info

redirecting (type: modules) ansible.builtin.podman_image_info to containers.podman.podman_image_info

Im beginning to suspect all the swithcing is causing delay on my build pipeline

5

As far as I’m aware these notices / warnings are because you are not using fully qualified names for the modules / connection plugins / filters / tests? I doubt they cause much of a delay, but there are people here better qualified than me to answer this.

1 Like
6

That is my impression as well. If you don’t specify the FQCN of a plugin, it’s inferred as ansible.builtin.<plugin> and Ansible redirects to the “first match” in the event that more than one collection has a plugin by that name in the given context.

Redirection can also happen in deprecated FQCN’s that get moved/reorganized into different collections, but those are explicitly defined redirects, not “guesses.”