Galaxy-NG: Remote sync fails "No space left on device"

Get Help
1

In my galaxy-ng I want to use the “community” remote to sync two collections from galaxy.ansible.com.

In the remote setting I have set the following “YAML requirements”:

collections:
  - name: community.crypto
    version: ">=2.18.0"
  - name: community.general
    version: ">=8.5.0"

When starting the sync I see a lot of json files created in the pulp_storage/tmp directory.
After a few hours the “sync” task fails with “[Errno 28] No space left on device”.
I have 60GB of available disk space before I start the sync. During the sync it is filling up the whole disk with these json files.

I am running the pulp/galaxy:4.9.1 container image from the docker hub via docker-compose.

Have you encountered anything like this?

2

Well, I don’t expect 4 collections (2 versions of each) to fill up 60GB, so I’m wondering if your free space is in a different partition from where the sync is going? Or do you actually see the 60GB of free space filling up?

When you run the sync, how many artifacts does the task show it try to download?

image
image1898×935 69.8 KB

1 Like
3

I first thought so too, but I believe to have mounted the directories correctly.

Part of my docker-compose.yml:

services:
  galaxy_ng:
    image: pulp/galaxy:4.9.1
    ports:
      - "80"
    container_name: galaxy_ng
    volumes:
      - ./settings/settings.py:/etc/pulp/settings.py
      - /data/pulp_storage:/var/lib/pulp
      - /data/pgsql:/var/lib/pgsql
      - /etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:ro

After starting the sync, I can see how the pulp_storage/tmp is filling up the disk

root@galaxy:/data# du -h --max-depth=1
43G     ./pulp_storage
16K     ./lost+found
88M     ./pgsql
20K     ./certs
44G     .

root@galaxy:/data/pulp_storage# du -h --max-depth=1
80K     ./.ansible
43M     ./media
43G     ./tmp
12K     ./.local
40K     ./.cache
4.0K    ./scripts
44G     .

root@galaxy:/data/pulp_storage/tmp# find | wc -l
150459

In the GUI I just see “running” but no count.

4

Sorry @h-phil , I’m not sure what to tell you from here, since I don’t use the compose install. Did you follow the documentation? Docker compose environment - Galaxy NG

1 Like
5

@Denney-tech Thanks for the help. I actually found the problem. It is completely unrelated to the galaxy-ng or compose.

The sync was done through a proxy that mistakenly cached API calls and produced this behavior (Cache invalidation problem).

2 Likes