# Every SSH connection logs mux\_client\_read\_packet: read header failed: Broken pipe **URL:** \ **Category:** Ansible Project\ **Created:** [February 25, 2020, 2:01am UTC](https://forum.ansible.com/t/every-ssh-connection-logs-mux-client-read-packet-read-header-failed-broken-pipe/32320 "2020-02-25T02:01:06Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![John\_Bateman](https://avatars.discourse-cdn.com/v4/letter/j/f9ae1b/32.png) [@John\_Bateman](https://forum.ansible.com/u/John_Bateman)\ **Post date:** [February 25, 2020, 2:01am UTC](https://forum.ansible.com/t/every-ssh-connection-logs-mux-client-read-packet-read-header-failed-broken-pipe/32320/1 "2020-02-25T02:01:06Z")
Each SSH connection throughout every playbook execution ends with Broken Pipe error message `mux_client_read_packet: read header failed: Broken pipe`, seen with verbose logging enabled. This doesn’t cause the failure of a task and the connection can still be seen to return a 0 exit code and the playbook executes successfully. I’m not getting any performance improvements with pipelining enabled and I am curious if this would have a performance impact on ControlPersist and pipelining if connections are terminating like this? Full debug output here: [https://gist.github.com/johnbateman/c2257c750798d1f48adc95deb66e3b61#file-gistfile1-txt](https://gist.github.com/johnbateman/c2257c750798d1f48adc95deb66e3b61#file-gistfile1-txt) ` ```auto ansible 2.9.4 config file = /etc/ansible/ansible.cfg configured module search path = [u'/home/vagrant/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.7/site-packages/ansible executable location = /usr/bin/ansible python version = 2.7.5 (default, Aug 7 2019, 00:51:29) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] ``` ` ---
**Author:** ![John\_Bateman](https://avatars.discourse-cdn.com/v4/letter/j/f9ae1b/32.png) [@John\_Bateman](https://forum.ansible.com/u/John_Bateman)\ **Post date:** [March 4, 2020, 2:46am UTC](https://forum.ansible.com/t/every-ssh-connection-logs-mux-client-read-packet-read-header-failed-broken-pipe/32320/2 "2020-03-04T02:46:30Z")
Does anyone have any ideas on this. Can anyone else verify the same behaviour? ---
**Author:** ![cassell](https://avatars.discourse-cdn.com/v4/letter/c/ecae2f/32.png) [@cassell](https://forum.ansible.com/u/cassell)\ **Post date:** [March 4, 2020, 5:31am UTC](https://forum.ansible.com/t/every-ssh-connection-logs-mux-client-read-packet-read-header-failed-broken-pipe/32320/3 "2020-03-04T05:31:19Z")
> Each SSH connection throughout every playbook execution ends with > Broken Pipe error message `mux\_client\_read\_packet: read header failed: > Broken pipe`, seen with verbose logging enabled. This doesn't cause the > failure of a task and the connection can still be seen to return a 0 > exit code and the playbook executes successfully. I'm not getting any > performance improvements with pipelining enabled and I am curious if > this would have a performance impact on ControlPersist and pipelining > if connections are terminating like this? > > Full debug output here: > [https://gist.github.com/johnbateman/c2257c750798d1f48adc95deb66e3b61#file-gistfile1-txt](https://gist.github.com/johnbateman/c2257c750798d1f48adc95deb66e3b61#file-gistfile1-txt) Maybe try just 3 v's... I got lost in the output. V/r James Cassell ---
**Author:** ![Mark\_Hansen](https://avatars.discourse-cdn.com/v4/letter/m/9fc29f/32.png) [@Mark\_Hansen](https://forum.ansible.com/u/Mark_Hansen)\ **Post date:** [April 10, 2020, 12:29pm UTC](https://forum.ansible.com/t/every-ssh-connection-logs-mux-client-read-packet-read-header-failed-broken-pipe/32320/4 "2020-04-10T12:29:36Z")
I’m also observing this problem too. ansible-playbook 2.9.5 config file = /Users/mark/projects/pis/ansible.cfg configured module search path = [‘/Users/mark/.ansible/plugins/modules’, ‘/usr/share/ansible/plugins/modules’] ansible python module location = /Users/mark/Library/Python/3.7/lib/python/site-packages/ansible executable location = /Users/mark/Library/Python/3.7/bin/ansible-playbook python version = 3.7.6 (default, Dec 30 2019, 19:38:26) [Clang 11.0.0 (clang-1100.0.33.16)] Every task execution notes in the logs debug3: mux\_client\_read\_packet: read header failed: Broken pipe ---
**Author:** ![Mark\_Hansen](https://avatars.discourse-cdn.com/v4/letter/m/9fc29f/32.png) [@Mark\_Hansen](https://forum.ansible.com/u/Mark_Hansen)\ **Post date:** [April 10, 2020, 12:31pm UTC](https://forum.ansible.com/t/every-ssh-connection-logs-mux-client-read-packet-read-header-failed-broken-pipe/32320/5 "2020-04-10T12:31:12Z")
Oh, and here’s my ansible.cfg > cat ansible.cfg > [ssh\_connection] > pipelining = True # ssh\_args = -C -o ControlMaster=auto -o ControlPersist=30m # [SSH connection sharing is broken · Issue #26359 · ansible/ansible · GitHub](https://github.com/ansible/ansible/issues/26359) [defaults] # Defaults to /usr/bin/python otherwise interpreter\_python = /usr/bin/python3 # [Profiling Ansible playbooks | Sketching Developer](https://sketchingdev.co.uk/blog/profiling-ansible-playbooks-to-csv.html) callback\_whitelist = profile\_tasks ---
**Author:** ![Mark\_Hansen](https://avatars.discourse-cdn.com/v4/letter/m/9fc29f/32.png) [@Mark\_Hansen](https://forum.ansible.com/u/Mark_Hansen)\ **Post date:** [April 10, 2020, 12:52pm UTC](https://forum.ansible.com/t/every-ssh-connection-logs-mux-client-read-packet-read-header-failed-broken-pipe/32320/6 "2020-04-10T12:52:34Z")
I’m investigating a bit further, I see the SSH command that ansible is executing (look for the line starting with “SSH: EXEC”, and I try running that in isolation: ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“pi”’ -o ConnectTimeout=10 -o ControlPath=/Users/mark/.ansible/cp/c4fffefc6d pi4b ‘/bin/sh -c “/usr/bin/python3 && sleep 0”’ Then the shell will hang waiting for you to enter a python command (it’s running /usr/bin/python3). you can type print(“hello”) to see this - it prints “hello”. print(‘hello’) ^D (eof) hello debug3: mux\_client\_read\_packet: read header failed: Broken pipe debug2: Received exit status from master 0 If I run this in multiple terminals, I can see that the error “debug3: mux\_client\_read\_packet: read header failed: Broken pipe” is an error being printed by SSH, not by ansible itself. It also appears that SSH is actually reusing the multiplexed connection - if I run this command from multiple terminals, the first terminal shows status messages about the connection for the other terminals. ---
**Author:** ![Mark\_Hansen](https://avatars.discourse-cdn.com/v4/letter/m/9fc29f/32.png) [@Mark\_Hansen](https://forum.ansible.com/u/Mark_Hansen)\ **Post date:** [April 10, 2020, 12:59pm UTC](https://forum.ansible.com/t/every-ssh-connection-logs-mux-client-read-packet-read-header-failed-broken-pipe/32320/7 "2020-04-10T12:59:51Z")
I tried taking a packet capture with Wireshark just to be double-sure that Ansible was reusing the same connection, and Ansible was. I ran wireshark, limited capture to “tcp.port == 22”, and then went to Menu → Statistics → Conversations → TCP, and I observe only one long-lived TCP SSH connection per host. Which is to say, I don’t think this “broken pipe” error is breaking connection pooling.