SemLock / No space left on device

Archives Ansible Project
1

This is not a Ansible problem but its stopping me from using Ansible on my Mac, if I reboot the system the problem goes away for about a day.

I am unable to find much information about this problem this is the most relevant information I could find. https://stackoverflow.com/questions/28788319/no-space-left-on-device-when-creating-semaphore

Any help would be great!

The error I get from Ansible is…

ansible-playbook playbook.yml -vvv

Using /etc/ansible/ansible.cfg as config file

[WARNING]: provided hosts list is empty, only localhost is available

ERROR! Unexpected Exception: [Errno 28] No space left on device

the full traceback was:

Traceback (most recent call last):

File “/usr/local/bin/ansible-playbook”, line 103, in

exit_code = cli.run()

File “/Library/Python/2.7/site-packages/ansible/cli/playbook.py”, line 154, in run

pbex = PlaybookExecutor(playbooks=self.args, inventory=inventory, variable_manager=variable_manager, loader=loader, options=self.options, passwords=passwords)

File “/Library/Python/2.7/site-packages/ansible/executor/playbook_executor.py”, line 59, in init

self._tqm = TaskQueueManager(inventory=inventory, variable_manager=variable_manager, loader=loader, options=options, passwords=self.passwords)

File “/Library/Python/2.7/site-packages/ansible/executor/task_queue_manager.py”, line 101, in init

self._final_q = multiprocessing.Queue()

File “/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/init.py”, line 218, in Queue

return Queue(maxsize)

File “/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/queues.py”, line 63, in init

self._rlock = Lock()

File “/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/synchronize.py”, line 147, in init

SemLock.init(self, SEMAPHORE, 1, 1)

File “/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/synchronize.py”, line 75, in init

sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)

OSError: [Errno 28] No space left on device

Running the following using any version of Python returns the following error.

>python -c ‘import multiprocessing; multiprocessing.Semaphore()’

Traceback (most recent call last):

File “”, line 1, in

File “/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/init.py”, line 197, in Semaphore

return Semaphore(value)

File “/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/synchronize.py”, line 111, in init

SemLock.init(self, SEMAPHORE, value, SEM_VALUE_MAX)

File “/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/synchronize.py”, line 75, in init

sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)

OSError: [Errno 28] No space left on device

2

I partially found the answer to my problem and hopefully the answer will be of use to somebody. The problem is I am running out of semaphores and I was using at the system V tools to view semaphores and not the POSIX command set.

Unfortunately I am still unable to figure out what is using the semaphores and the usage continues to grow the default is 1000 and I have had to raise the value twice.

sudo sysctl -w kern.posix.sem.max=20000sudo zprint semaphores

Dave