Unable to reuse ssh connections in Ansible despite PIPELINING = True

Archives Ansible Project
1

My corporate firewall policy allows only 20 connections per minute 60 seconds between the same source and destinations.

Owing to this the ansible play hangs after a while.

I would like multiple tasks to use the same ssh session rather than creating new sessions. For this purpose i set the below pipelining = True in the local folder ansible.cfg as well as in the command line.

cat /opt/automation/startservices/ansible.cfg

[defaults]
host_key_checking = False
gathering = smart
[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=600s
control_path = %(directory)s/%%h-%%r
pipelining = True

ANSIBLE_SSH_PIPELINING=0 ansible-playbook -i /opt/automation/startservices/finalallmw.hosts /opt/automation/startservices/va_action.yml -e ‘{ dest_host: myremotehost7 }’ -e dest_user=oracle

The playbook is too big to be shared here but it is this task which loops and this is where it hangs due to more than 20 ssh connections in 60 seconds.

171 - name: Copying from “{{ inventory_hostname }}” to this ansible server.
172 synchronize:
173 src: “{{ item.path }}”
174 dest: “{{ playbook_dir }}/homedirbackup/{{ inventory_hostname }}/{{ dtime }}/”
175 mode: pull
176 copy_links: yes
177 with_items:
178 - “{{ to_copy.files }}”

With the pipelining settings set; my play still hangs after 20 connections.

Below are the playbook settings:

45 hosts: “{{ groups[‘dest_nodes’] | default(groups[‘all’]) }}”
46 user: “{{ USER | default(dest_user) }}”
47 any_errors_fatal: True
49 gather_facts: false
51 tags: always
52
53 vars:
54 ansible_host_key_checking: false
55 ansible_ssh_extra_args: -o StrictHostKeyChecking=no -o ConnectionAttempts=5

Can you please suggest any solution to the issue on the ansible side where all tasks use the same ssh session and is pipelining not working here?

2

My corporate firewall policy allows only 20 connections per minute 60 seconds between the same source and destinations.

Sounds to me like a ridiculous policy. Ask for an exception instead of trying to throttle Ansible.

Regads
       Racke

3

I agree. This would cause problems just for a large number of standard
websites, unless your IT networking people are relying on browsers using
persistent connections, and even then, static content, images, and dynamic
content are often going to be supplied by different parts of a CDN.

Antony.

4

Hi,

Did you find a solution for this?

I don’t think pipelining will actually result in reusing the same SSH session across the play. It will reduce the number of SSH sessions per task, but I think it will still result in many new SSH sessions.

I can’t figure out how to reuse sessions for a non-SSH connector.

Regards,
Matt

5

A few things:
- pipelining is not about connection reuse but about writing to disk
- for ssh connection plugin, the control persist settings are what
reuse connections/authentication, but not sessions
- the synchronize action does it's own connection handling

6

Consider you might have conflicting settings in your ssh config (~/.ssh/config and/or /etc/ssh/ssh_config)?

Here’s what I have…
Host *
ControlMaster auto
ControlPath ~/.ansible/cp/%h-%r

ControlPersist 10m

control_path probably needs to match ControlPath. Not sure about the others.

7
  • for ssh connection plugin, the control persist settings are what reuse connections/authentication, but not sessions

Is there a mechanism to re-use sessions? Or a mechanism to re-use connections for non-ssh connection plugins?

Thanks,
Matt

8

Kind of, there is ansible-connection but it can be complicated. I've
been toying around with the idea of a 'persistent: no|yes' keyword to
allow for the use/reuse of persistent sessions. This would first look
at the connection plugin for support and fallback to a connection
manager+ansible-connection on controller otherwise.

9

Why would you need to check whether the connection plugin supports such a new feature? ConnectionBase already has methods for _connect, connected, reset and close, separate from exec_command, put_file and fetch_file
So wouldn’t this work for all connection plugins if Ansible just starts calling exec_command and put_file multiple times per connection?

10

Ansible already does this per per task, this still just uses 'shared
auth' in the case of ssh, not full session (unlike winrm/prsp). In any
case this feature would be 'across tasks' or block of tasks.