Running containers and building images in AWX-EE

Archives AWX Project
,
1

We have some Ansible playbooks that we’ve been using to build images and run containers to execute some tasks (not using AWX).

Recently, we’ve decided to use AWX v21 and tried to run these playbooks from there but have been getting errors that we couldn’t figure out when we tried to run podman.

The AWX is deployed on EKS, and we have a custom AWX EE with podman installed and configured /etc/subuid /etc/subgid for ‘runner’.

The error I’m getting is as follows when I run podman run --rm busybox echo ‘hello’ from my job.

Resolved “busybox” as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/busybox:latest…
Getting image source signatures
Copying blob sha256:19d511225f94f9b5cbf3836eb02b5273c01b95da50735742560e3e45b8c8bfcc
Copying blob sha256:19d511225f94f9b5cbf3836eb02b5273c01b95da50735742560e3e45b8c8bfcc
Copying config sha256:62aedd01bd8520c43d06b09f7a0f67ba9720bdc04631a8242c65ea995f3ecac8
Writing manifest to image destination
Storing signatures
time=“2022-07-13T16:00:58Z” level=warning msg=“failed to set net.ipv6.conf.default.accept_dad sysctl: open /proc/sys/net/ipv6/conf/default/accept_dad: read-only file system”
time=“2022-07-13T16:00:58Z” level=error msg=“Unmounting /home/runner/.local/share/containers/storage/overlay/ef4f79040ba910a4219a4d857c2ec9aff76f46f546e78ecfec71e370e378c07f/merged: invalid argument”
time=“2022-07-13T16:00:59Z” level=error msg=“Preparing container b347a64b9d428f1c01c8d96c1c62322dccce1f5996f832db9462f63e44276b8a: /usr/bin/slirp4netns failed: "open(\"/dev/net/tun\"): No such file or directory\nWARNING: Support for seccomp is experimental\nWARNING: Support for IPv6 is experimental\nchild failed(1)\nWARNING: Support for seccomp is experimental\nWARNING: Support for IPv6 is experimental\n"”
Error: error mounting storage for container b347a64b9d428f1c01c8d96c1c62322dccce1f5996f832db9462f63e44276b8a: creating overlay mount to /home/runner/.local/share/containers/storage/overlay/ef4f79040ba910a4219a4d857c2ec9aff76f46f546e78ecfec71e370e378c07f/merged, mount_data=“,lowerdir=/home/runner/.local/share/containers/storage/overlay/l/2VRGIUCXCOXP63FLF7J2DTBUKC,upperdir=/home/runner/.local/share/containers/storage/overlay/ef4f79040ba910a4219a4d857c2ec9aff76f46f546e78ecfec71e370e378c07f/diff,workdir=/home/runner/.local/share/containers/storage/overlay/ef4f79040ba910a4219a4d857c2ec9aff76f46f546e78ecfec71e370e378c07f/work,volatile”: using mount program /usr/bin/fuse-overlayfs: fuse: device not found, try ‘modprobe fuse’ first
fuse-overlayfs: cannot mount: No such file or directory
: exit status 1

Any help/information to resolve this would be much appreciated.

2

Hi there,

You did not say whether you were using podman before so it’s hard to pinpoint what your issue is.

From the error message, it looks like you are having issues w/ fuse itself on the machine you are running podman (I guess your AWX EE but you did not mention that explicitely).

I would check the filesystem configuration for podman to check this is in line with what you want (e.g. using FUSE in overlayfs mode for your container).

HTH,

Vincent

3

Hi Vincent,

Thanks for the response.

To clarify, all I’m trying to do right now is to test if I can run a container with podman on my AWX EE.

My AWX EE is built by adding the following lines to the Containerfile file from https://github.com/ansible/awx-ee:

RUN yum install -y podman

RUN echo runner:10000:65536 > /etc/subuid
RUN echo runner:10000:65536 > /etc/subgid

Is there a reference/guide on how to build a proper AWX EE image which can run containers via podman? I can check the filesystem configuration as you suggested, but I wouldn’t know what I’m supposed to be checking.

Thanks,

NG

4

Hi NG,

Please see my answers below.

Hi Vincent,

Thanks for the response.

To clarify, all I’m trying to do right now is to test if I can run a container with podman on my AWX EE.

Understood, thanks for precising that. This is thus independent of AWX itself at this point

My AWX EE is built by adding the following lines to the Containerfile file from https://github.com/ansible/awx-ee:

RUN yum install -y podman

RUN echo runner:10000:65536 > /etc/subuid
RUN echo runner:10000:65536 > /etc/subgid

Is there a reference/guide on how to build a proper AWX EE image which can run containers via podman? I can check the filesystem configuration as you suggested, but I wouldn’t know what I’m supposed to be checking

There is no guide per se on doing that I think. The automation controller documentation do not detail this part, I suggest checking the source of AWX itself to see if you can find some more things.

At this point, I would:

  1. Share the exact commandline + configuraiton you are using to run your test job w/ podman
  2. Build a container using the same base image and debug until you get podman working. Then move that setup into the AWX EE and see if it also works there. It indeed. looks to me more like a podman config issue

HTH,

VIncent