bad PATH when running "apt" module

I’m trying to install a package on a system running a Debian Jessie-based distribution called OSMC.

This task (run with ansible_sudo == true):

  • name: ensure Vim is installed

apt: name=vim state=present

causes this to happen:

TASK: [vim | ensure Vim is installed] *****************************************

failed: [XXX] => {“failed”: true}

stderr: E: Sub-process /usr/bin/dpkg returned an error code (2)

stdout: Reading package lists…

dpkg: warning: ‘ldconfig’ not found in PATH or not executable

dpkg: warning: ‘start-stop-daemon’ not found in PATH or not executable

dpkg: error: 2 expected programs not found in PATH or not executable

Note: root’s PATH should usually contain /usr/local/sbin, /usr/sbin and /sbin

msg: ‘/usr/bin/apt-get -y -o “Dpkg::Options::=–force-confdef” -o “Dpkg::Options::=–force-confold” install ‘vim’’ failed: E: Sub-process /usr/bin/dpkg returned an error code (2)

It seems that something is mangling the PATH before apt-get is called, because if I SSH manually into this system:

$ which ldconfig start-stop-daemon

/sbin/ldconfig

/sbin/start-stop-daemon

$ echo $PATH

/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/opt/vc/bin

$ sudo sh -c ‘echo $PATH’

/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/opt/vc/bin

the PATH is correct and I have no problem installing the package manually.

What can I do to improve this situation?

When you gather facts on the machine, you should be able to see the
'PATH' that ansible sees under ansible_env, ansible uses it's own path
list to find apt and dpkg, but that should not influence the other
tools, which ansible is not using directly.

When you gather facts on the machine, you should be able to see the
‘PATH’ that ansible sees under ansible_env, ansible uses it’s own path
list to find apt and dpkg, but that should not influence the other
tools, which ansible is not using directly.

Thanks for your response. If I understand you correctly, I can also do this to see what PATH Ansible is using:

$ ansible -m setup XXX

XXX | success >> {

“ansible_facts”: {

“ansible_env”: {

“PATH”: “/usr/local/bin:/usr/bin:/bin:/usr/games”,

“USERNAME”: “root”

},

},

“changed”: false

}

The USERNAME of “root” tells me that Ansible is correctly sudo-ing before running the setup, so I’d expect it to be doing the same before apt-ing.

If this is the PATH setting when Ansible runs apt-get though, it’s no surprise to me that it’s failing when it tries to hit things in /sbin.

What could be responsible for this discrepancy in the PATHs based on how the system is being accessed? Is this likely to be a system configuration issue or an Ansible (v1.9.1) issue?

The first one I would look at is sudo (/etc/sudoers), as it normally
resets all env variables for a user.

The first one I would look at is sudo (/etc/sudoers), as it normally
resets all env variables for a user.

The contents of my /etc/sudoers and /etc/sudoers.d/* files are:

Defaults env_reset

Defaults mail_badpass

Defaults secure_path=“/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin”

root ALL=(ALL:ALL) ALL

%sudo ALL=(ALL:ALL) ALL

Defaults env_keep += “RPI_UPDATE_UNSUPPORTED”

Defaults !secure_path

I can’t really see an issue there, but I’m not very experienced with these files. I note though that if I SSH into this machine manually, the PATHs are still okay:

$ echo $PATH

/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/opt/vc/bin

$ sudo su

echo $PATH

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

that is, they still have /sbin in them both before and after sudo-ing… Is my sudo setup still a possible culprit?

try: sudo sh -c 'echo $PATH'

su can readd path info, the above is closer to what ansible does (it
uses sudo -u <user> not sudo su).

$ sudo sh -c ‘echo $PATH’
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/opt/vc/bin

try: sudo sh -c ‘echo $PATH’

su can readd path info, the above is closer to what ansible does (it
uses sudo -u not sudo su).

Done:

$ sudo sh -c ‘echo $PATH’
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/opt/vc/bin

Does this rule out problems with my sudo setup? Any other ideas?

I’ve been unable to resolve this issue with the ideas presented so far. In the absence of any further responses, should I consider this an Ansible bug and file a bug report?

i'm unable to reproduce the issue, the path seems fine, not sure if
its a bug or not, but without a way to reproduce the issue I'm not
sure how it will get resolved.

Hi Alex,

if you are looking for problems with sudo you can login as your ansible user and issue “sudo -l”. This will list all commands your ansible user is allowed to use.

From your sudo config files, %sudo group can fire all kind of commands, but needs a password. That’s by intention?

Regards,
Marcus

Would giving you SSH access to this particular machine be useful?

if you are looking for problems with sudo you can login as your ansible user and issue “sudo -l”. This will list all commands your ansible user is allowed to use.

The output of this command on the affected machine is as follows:

$ sudo -l
Matching Defaults entries for osmc on osmc:
env_reset, mail_badpass, secure_path=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin, env_keep+=RPI_UPDATE_UNSUPPORTED, !secure_path

User osmc may run the following commands on osmc:
(root) NOPASSWD: ALL

To me, this indicates that after running sudo, the PATH should still contain /sbin. My problems with Ansible’s apt module seem to strongly indicate that this is only not the case when Ansible is involved. Although I appreciate that this is not easy to replicate on another machine, I still feel that on the current balance of probabilities, this indicates an Ansible bug. I’m happy to provide Ansible developers with SSH access to this particular machine for further diagnosis if it’s not possible to suggest any other solutions at this time.

From your sudo config files, %sudo group can fire all kind of commands, but needs a password. That’s by intention?

This is a largely pre-configured system based off another distribution, so it wouldn’t surprise me if this definition is an unintended remnant from the base distribution. The Ansible user is not part of the sudo group. so I don’t believe it should have any effect.

I hate bumping my own posts without contributing any more information, but I’m totally stuck on this issue and I can’t progress without assistance.

Can someone please provide me with some more direction towards resolving this problem?

sorry to say, I'm stumped here, i have not been able to reproduce your
issue and really don't understand how it can happen with the
information given.

I tried running this very verbose command:

ansible -vvv -m setup XXX

which caused the following to be output (among other things):

EXEC sshpass -d6 ssh -C -tt -v -o ControlMaster=auto -o ControlPersist=60s -o ControlPath=“…/.ansible/cp/ansible-ssh-%h-%p-%r” -o GSSAPIAuthentication=no -o PubkeyAuthentication=no -o User=osmc -o ConnectTimeout=10 XXX /bin/sh -c ‘sudo -k && sudo -H -S -p “[sudo via ansible, key=xidpkckjjfdcsgiewleebdlstyiyzpad] password: " -u root /bin/sh -c '”’“‘echo BECOME-SUCCESS-xidpkckjjfdcsgiewleebdlstyiyzpad; LANG=C LC_CTYPE=C /usr/bin/python /home/osmc/.ansible/tmp/ansible-tmp-1431605612.35-230680545785241/setup; rm -rf /home/osmc/.ansible/tmp/ansible-tmp-1431605612.35-230680545785241/ >/dev/null 2>&1’”‘"’’

which prompted me to cut out Ansible altogether and just try this:

$ ssh osmc@XXX /bin/sh -c ‘“echo $PATH”’
OSMC: Open Source Media Center
osmc@XXX’s password:
/usr/local/bin:/usr/bin:/bin:/usr/games

$ ssh osmc@XXX /usr/bin/sudo -u root /bin/sh -c ‘“echo $PATH”’
OSMC: Open Source Media Center
osmc@XXX’s password:
/usr/local/bin:/usr/bin:/bin:/usr/games

which is a very different PATH to when I log in interactively:

$ ssh osmc@XXX
OSMC: Open Source Media Center
osmc@XXX’s password:
osmc@XXX:~$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/opt/vc/bin
osmc@XXX:~$ sudo echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/opt/vc/bin

So on the bright side, it doesn’t seem to be an Ansible problem after all, which means it’s probably out of scope for this discussion group. Time to read up on the differences between interactive, non-interactive, login and non-login bashsessions, it seems…

Thanks anyway for all of your attention to this point. I’m sorry to have wasted your time.

Did you find a solution to this by any chance?

I’ve got exactly the same problem and am beginning to get mighty pissed
at the amount of useless yak-shaving that isn’t even working.

No matter what I try, it looks like the secure_path in the sudoers file
is ignored somewhere down the call stack.

I did find this reference: https://wiki.debian.org/sudo#PATH_not_set.
But, I don’t seem to have any other sudoers file and this isn’t an upgrade.

Completely stumped and reaching the end of my patience with this
(been at this one issue for literally a week now)

Any help HUGELY appreciated

Thanks
Alastair

Hi there,

something like this might work just fine…

  • name: upgrade all installed packages apt: update_cache=yes upgrade=yes environment: PATH: /bin:/sbin:/usr/bin:/usr/sbin

Cheers
Ralph

I put in this task to make sure secure_path is used. It is now part of my “set the node up for use with ansible” playbook:

  • name: Special for osmc. Make sure secure_path is honored
    file: path=/etc/sudoers.d/osmc-no-secure-path state=absent

There is a discussion on this in the osmc forum where one of the developers says this is not something they support officially but there shouldn’t be any issues and so far I haven’t had any. (I had this already set up before the last major osmc Update)
https://discourse.osmc.tv/t/system-path-broken-when-running-a-non-interactive-sudo-command/4167/6

There is also another workaround in the thread where the PATH env is set as part of the task

I have this as part of my “set node up for ansible” playbook.

  • name: Special for osmc. Make sure secure_path is honored
    file: path=/etc/sudoers.d/osmc-no-secure-path state=absent

For more info look here:
https://discourse.osmc.tv/t/system-path-broken-when-running-a-non-interactive-sudo-command/4167/6

Worked for me so far and I don’t have to set env vars during apt tasks (useful because I don’t want to alter my site-yml playbook for just the osmc node).