setting explicit remote shell

hi all,

strange little problem here, so i have this quite quirky role which removes all rpm’s from an aix system and rolls out a whole new rpm distribution. this also involves
removal of the default python and installing a fallback python for ansible, but this all works and is already sorted out.

per default the root user, which is used by ansible as the remote user has bash set as default shell. this is changed during play to ksh via the chsh command.
the following environment/variables are set in the ansible role/play level.

`
ansible_shell_executable: “/usr/bin/ksh”
ansible_shell_interpreter: “/usr/bin/ksh”
ansible_python_interpreter: “/opt/bin/python”
SHELL: “/usr/bin/ksh”
PATH: “/opt/bin:/opt/freeware/bin:{{ ansible_env.PATH }}”
LIBPATH: “/opt/lib:/usr/lib:/lib:/opt/freeware/lib”

`

anyhow, somehing inside ansible persits using /usr/bin/bash which make the whole play fail.

TASK [do_aixtoolbox_migration : destroy all rpm packages] ************************************************************************************************************** task path: /etc/ansible/aix/roles/do_aixtoolbox_migration/tasks/main.yml:36 <172.17.14.13> ESTABLISH SSH CONNECTION FOR USER: root <172.17.14.13> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=20 -o ControlPath=/tmp/ansible-ssh-%h-%p-%r 172.17.14.13 '/usr/bin/ksh -c '"'"'echo ~root && sleep 0'"'"'' <172.17.14.13> (0, '/root\n', '') <172.17.14.13> ESTABLISH SSH CONNECTION FOR USER: root <172.17.14.13> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=20 -o ControlPath=/tmp/ansible-ssh-%h-%p-%r 172.17.14.13 '/usr/bin/ksh -c '"'"'( umask 77 && mkdir -p " echo /root/.ansible/tmp/ansible-tmp-1587558405.77-115997468659119 " && echo ansible-tmp-1587558405.77-115997468659119=" echo /root/.ansible/tmp/ansible-tmp-1587558405.77-115997468659119 `" ) && sleep 0’“'”‘’
<172.17.14.13> (0, ‘ansible-tmp-1587558405.77-115997468659119=/root/.ansible/tmp/ansible-tmp-1587558405.77-115997468659119\n’, ‘’)
Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py
<172.17.14.13> PUT /root/.ansible/tmp/ansible-local-28818rmvv6A/tmp0Mbfvc TO /root/.ansible/tmp/ansible-tmp-1587558405.77-115997468659119/AnsiballZ_command.py
<172.17.14.13> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“root”’ -o ConnectTimeout=20 -o ControlPath=/tmp/ansible-ssh-%h-%p-%r ‘[172.17.14.13]’
<172.17.14.13> (0, ‘sftp> put /root/.ansible/tmp/ansible-local-28818rmvv6A/tmp0Mbfvc /root/.ansible/tmp/ansible-tmp-1587558405.77-115997468659119/AnsiballZ_command.py\n’, ‘’)
<172.17.14.13> ESTABLISH SSH CONNECTION FOR USER: root
<172.17.14.13> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“root”’ -o ConnectTimeout=20 -o ControlPath=/tmp/ansible-ssh-%h-%p-%r 172.17.14.13 ‘/usr/bin/ksh -c ‘"’“‘chmod u+x /root/.ansible/tmp/ansible-tmp-1587558405.77-115997468659119/ /root/.ansible/tmp/ansible-tmp-1587558405.77-115997468659119/AnsiballZ_command.py && sleep 0’”’“‘’
<172.17.14.13> (0, ‘’, ‘’)
<172.17.14.13> ESTABLISH SSH CONNECTION FOR USER: root
<172.17.14.13> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“root”’ -o ConnectTimeout=20 -o ControlPath=/tmp/ansible-ssh-%h-%p-%r -tt 172.17.14.13 '/usr/bin/ksh -c '”‘“‘HTTP_PROXY=http://proxy.sozvers.at:8080 HTTPS_PROXY=http://proxy.sozvers.at:8080 no_proxy=localhost,127.0.0.1,nimmsag_a,.itsv.at,.sozvers.at http_proxy=http://proxy.sozvers.at:8080 ftp_proxy=http://proxy.sozvers.at:8181 LIBPATH=/opt/lib:/usr/lib:/lib:/opt/freeware/lib https_proxy=http://proxy.sozvers.at:8080 SHELL=/usr/bin/ksh FTP_PROXY=http://proxy.sozvers.at:8181 /opt/bin/python /root/.ansible/tmp/ansible-tmp-1587558405.77-115997468659119/AnsiballZ_command.py && sleep 0’”’“‘’
<172.17.14.13> (0, ‘\r\n{“changed”: true, “end”: “2020-04-22 14:26:59.034312”, “stdout”: “Preparing packages…\nyum-3.4.3-7.noarch\nyum-metadata-parser-1.1.4-2.ppc\npython-tools-2.7.10-1.ppc\npysqlite-1.1.7-2.ppc\nsqlite-3.15.2-1.ppc\npython-urlgrabber-3.10.1-1.noarch\npython-pycurl-7.19.3-1.ppc\ncurl-7.52.1-1.ppc\npython-devel-2.7.10-1.ppc\nca-certificates-2016.10.7-2.ppc\npython-iniparse-0.4-1.noarch\npython-2.7.10-1.ppc\nreadline-6.1-2.ppc\nlibxml2-2.9.9-1.ppc\nglib2-2.56.1-2.ppc\nlibiconv-1.14-2.ppc\ngettext-0.19.8.1-5.ppc\nlibunistring-0.9.9-2.ppc\ninfo-6.6-2.ppc\nbash-5.0-1.ppc\nxz-libs-5.2.4-1.ppc\nncurses-6.2-1.ppc\nlibstdc+±8.3.0-2.ppc\nlibffi-3.2.1-3.ppc\nlibgcc-8.3.0-2.ppc\nzlib-1.2.11-1.ppc\ndb-4.8.24-3.ppc\ngdbm-1.8.3-5.ppc\nAIX-rpm-7.2.3.15-1.ppc\nSUCCESSES\n---------\n Filesets listed in this section passed pre-deinstall verification\n and will be removed.\n\n Selected Filesets\n -----------------\n rpm.rte 4.13.0.10 # RPM Package Manager\n\n << End of Success Section >>\n\nrmdir: Directory /opt/freeware/etc/httpd/conf is not empty.\nrmdir: Directory /opt/freeware/etc/httpd is not empty.\nrmdir: Directory /opt/freeware/etc/yum is not empty.\nrmdir: Directory /opt/freeware/etc is not empty.\nrmdir: Directory /opt/freeware/lib/python2.7/site-packages/rpm is not empty.\nrmdir: Directory /opt/freeware/lib/python2.7/site-packages is not empty.\nrmdir: Directory /opt/freeware/lib/python2.7 is not empty.\nrmdir: Directory /opt/freeware/lib is not empty.\nrmdir: Directory /opt/freeware/packages is not empty.\nrmdir: Directory /opt/freeware/share/info is not empty.\nrmdir: Directory /opt/freeware/share is not empty.\nrmdir: Directory /opt/freeware is not empty.\n\nYour rpms have been removed, but the /opt/freeware and /usr/opt/freeware\ndirectories (one is just a symbolic link) have been left present.\nYou may want to look through these directories for any files you may have\ncreated yourself. If there is nothing that you want to save, you can now\nremove these directories and any remaining contents within.”, “cmd”: “/tmp/destroyRPMS -f”, “rc”: 0, “start”: “2020-04-22 14:26:46.211237”, “stderr”: "warning: /opt/freeware/etc/yum/yum.conf saved as /opt/freeware/etc/yum/yum.conf.rpmsave\nCould not load program /sbin/install-info:\nCould not load module /opt/freeware/lib64/libintl.a(libintl.so.8).\n\tDependent module /opt/freeware/lib/libiconv.a(libiconv.so.2) could not be loaded.\nCould not load module /opt/freeware/lib/libiconv.a(libiconv.so.2).\nSystem error: No such file or directory\nCould not load module install-info_64.\n\tDependent module /opt/freeware/lib64/libintl.a(libintl.so.8) could not be loaded.\nCould not load program /sbin/install-info:\n\tDependent module /opt/freeware/lib64/libintl.a(libintl.so.8) could not be loaded.\nCould not load module /opt/freeware/lib64/libintl.a(libintl.so.8).\nSystem error: No such file or directory\n±----------------------------------------------------------------------------+\n\t\t Pre-deinstall Verification…\n±----------------------------------------------------------------------------+\nVerifying selections…done\nVerifying requisites…done\nResults…\n\nFILESET STATISTICS \n------------------\n 1 Selected to be deinstalled, of which:\n 1 Passed pre-deinstall verification\n ----\n 1 Total to be deinstalled\n\n±----------------------------------------------------------------------------+\n\t\t\t Deinstalling Software…\n±----------------------------------------------------------------------------+\n\ninstallp: DEINSTALLING software for:\n\trpm.rte 4.13.0.10\n\nFinished processing all filesets. (Total time: 4 secs).\n\n±----------------------------------------------------------------------------+\n\t\t\t\tSummaries:\n±----------------------------------------------------------------------------+\n\nInstallation Summary\n--------------------\nName Level Part Event Result\n-------------------------------------------------------------------------------\nrpm.rte 4.13.0.10 ROOT DEINSTALL SUCCESS \nrpm.rte 4.13.0.10 USR DEINSTALL SUCCESS ", “delta”: “0:00:12.823075”, “invocation”: {“module_args”: {“creates”: null, “executable”: “/usr/bin/ksh”, “_uses_shell”: true, “strip_empty_ends”: true, “_raw_params”: “/tmp/destroyRPMS -f”, “removes”: null, “argv”: null, “warn”: true, “chdir”: null, “stdin_add_newline”: true, “stdin”: null}}}\r\n’, ‘Shared connection to 172.17.14.13 closed.\r\n’)
<172.17.14.13> ESTABLISH SSH CONNECTION FOR USER: root
<172.17.14.13> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“root”’ -o ConnectTimeout=20 -o ControlPath=/tmp/ansible-ssh-%h-%p-%r 172.17.14.13 '/usr/bin/ksh -c '”‘“‘rm -f -r /root/.ansible/tmp/ansible-tmp-1587558405.77-115997468659119/ > /dev/null 2>&1 && sleep 0’”’"‘’
<172.17.14.13> (1, ‘’, ‘/usr/bin/bash: No such file or directory\n’)
<172.17.14.13> Failed to connect to the host via ssh: /usr/bin/bash: No such file or directory
[WARNING]: Error deleting remote temporary files (rc: 1, stderr: /usr/bin/bash: No such file or directory })
changed: [AIXTEST01] => {
“changed”: true,
“cmd”: “/tmp/destroyRPMS -f”,
“delta”: “0:00:12.823075”,
“end”: “2020-04-22 14:26:59.034312”,
“failed_when_result”: false,
“invocation”: {
“module_args”: {
“_raw_params”: “/tmp/destroyRPMS -f”,
“_uses_shell”: true,
“argv”: null,
“chdir”: null,
“creates”: null,
“executable”: “/usr/bin/ksh”,
“removes”: null,
“stdin”: null,
“stdin_add_newline”: true,
“strip_empty_ends”: true,
“warn”: true
}
},
“rc”: 0,
“start”: “2020-04-22 14:26:46.211237”
}

STDOUT:

Preparing packages…
yum-3.4.3-7.noarch
yum-metadata-parser-1.1.4-2.ppc
python-tools-2.7.10-1.ppc
pysqlite-1.1.7-2.ppc
sqlite-3.15.2-1.ppc
python-urlgrabber-3.10.1-1.noarch
python-pycurl-7.19.3-1.ppc
curl-7.52.1-1.ppc
python-devel-2.7.10-1.ppc
ca-certificates-2016.10.7-2.ppc
python-iniparse-0.4-1.noarch
python-2.7.10-1.ppc
readline-6.1-2.ppc
libxml2-2.9.9-1.ppc
glib2-2.56.1-2.ppc
libiconv-1.14-2.ppc
gettext-0.19.8.1-5.ppc
libunistring-0.9.9-2.ppc
info-6.6-2.ppc
bash-5.0-1.ppc
xz-libs-5.2.4-1.ppc
ncurses-6.2-1.ppc
libstdc+±8.3.0-2.ppc
libffi-3.2.1-3.ppc
libgcc-8.3.0-2.ppc
zlib-1.2.11-1.ppc
db-4.8.24-3.ppc
gdbm-1.8.3-5.ppc
AIX-rpm-7.2.3.15-1.ppc
SUCCESSES

if you set these as environment variables, Ansible is not going to do
anything with them, they need to be variables.

ansible_shell_executable: "/usr/bin/ksh"
ansible_shell_interpreter: "/usr/bin/ksh"
ansible_python_interpreter: "/opt/bin/python"

Also the `environment:` keyword won't affect Ansible itself, any
environment setting for Ansible must be set BEFORE calling the ansible
executable.

In any case, the output seems to show you using `ksh` and the specific
python, so they must be set correctly, just posting the above to
clarify misconceptions.

I'm not sure what is causing the bash error, since the last command
(part of cleanup) clearly uses ksh also:

'/usr/bin/ksh -c '"'"'rm -f -r
/root/.ansible/tmp/ansible-tmp-1587558405.77-115997468659119/ >
/dev/null 2>&1 && sleep 0'"'"''

unless the ssh login itself is forcing bash usage. I suggest tracing
the process on the remote to try to find out what exactly is executed
to trigger that error, since the last ansible command clearly
specifies ksh.