We have a task in our Ansible based deploy which performs formatting of HDFS using the hdfs dfs -format command.
This command causes a MemoryError. We can see that there is a Python process on the target box which gobbles up ~32GB of memory and then the failure occurs.
How can we troubleshoot this further?
Also, any ideas/recommendations on how to manage Ansible’s memory usage? I’m peeking into ansible_memtotal_mb… any config settings to look at or adjust?
Thanks,
- Dmitry