Ansible fails with "Authentication or permission failure" due to "Too many links" in $HOME/.ansible/tmp/

Here is a sample failure message:

fatal: [u05] => Authentication or permission failure. In some cases, you may have been able to authenticate and did not have permissions on t
he remote directory. Consider changing the remote temp path in ansible.cfg to a path rooted in “/tmp”. Failed command was: mkdir -p $HOME/.ans
ible/tmp/ansible-1393461099.9-52166303344836 && echo $HOME/.ansible/tmp/ansible-1393461099.9-52166303344836, exited with result 1: mkdir: cann
ot create directory `/root/.ansible/tmp/ansible-1393461099.9-52166303344836’: Too many links

The reason is that /root/.ansible/tmp/ got filled with 32000 files. I am not sure why Ansible doesn’t clean up after itself, but that’s what happened on my system with basically default ansible.cfg (except parallelism was bumped up to 100). Ansible runs about every 5 minutes in my environment.

Thanks!
iordan

For the record, I’m using Ansible 1.4.5 built with “make deb” after a “git checkout v1.4.5” in the repository.

Was just going to ask what version and you beat me to it :slight_smile:

There was a problem with the copy module not removing tempfiles that was recently fixed (Feb 4), though we haven’t included this in a release – Ansible 1.5 is slated to be released by the end of this month.

The above problem with copy, I believe, is more likely to happen with recursive file copies.

Do you have any async tasks? Async can leave behind tempfiles as well and usually requires a bit of manual cleanup. There used to be a ticket to this effect though I’m not familiar with current status.

Hi Michael,

Was just going to ask what version and you beat me to it :slight_smile:

:slight_smile:

There was a problem with the copy module not removing tempfiles that was recently fixed (Feb 4), though we haven’t included this in a release – Ansible 1.5 is slated to be released by the end of this month.

The above problem with copy, I believe, is more likely to happen with recursive file copies.

I have a bunch of copies and a few of them are recursive indeed.

Do you have any async tasks? Async can leave behind tempfiles as well and usually requires a bit of manual cleanup. There used to be a ticket to this effect though I’m not familiar with current status.

Can you give me an example of an asynchronous task? I have a bunch of notify’s that get handled with separate handlers, but that’s about it.

Cheers!
iordan

Basically the “async” keyword in Ansible for very long running stuff.

http://docs.ansible.com/playbooks_async.html

The devel branch is pretty darn stable now if you want to try running an early copy (actually it usually is, but we may get more into refactoring/overhaul in the future, but that usually happens earlier in a release)