I’ve installed AWX on k3s. The server had almost 100G free space. It is not possible to allocate more storage to the server at the moment.
After a few days, the server’s storage gets full, causing AWX to stop working. How can this problem be resolved without changing the settings configured for templates and inventories?
Thank you
awx awx-operator #k3s kubernetes
Is it possible that your storage is being taken up by having too many days of job logs, event streams, etc? You can try to run the scheduled managed jobs and set the value of days to keep at 7 or so. We used to keep 30 days of job logs and stream events but it also added about 150G of data to our external postgres database. We brought the number of days to keep data to 7, ran the scheduled cleanup jobs and the database has been happy since.
I wish the AWX developers could figure out a way to possibly compress or otherwize minimize the data going into the database for historic lookups or perhaps keep it in an external file storage location like S3.
What storage is getting full? If the PVC is filling up, that’s one thing, but is AWX filling up the ephemeral storage? That’s something I’ve seen before, where someone forgot to extend the ephemeral storage partition of the underlying k3s host.