Help with awx-ee

Get Help
,
1

Hello,

I’m using awx 22.7 and getting some trouble to pull my awx-ee image from private registry.

My registry is on a forman server and presented on https.
I’m able to connect to it and pull image manually with podman commands.

When i try via awx i get some errors, 415 unsuppoted, what does it means:

 Normal   Pulling    11s   kubelet            Pulling image "myserver.lan/docker-awx-ee:latest"
  Warning  Failed     11s   kubelet            Failed to pull image "myserver.lan/docker-awx-ee:latest": rpc error: code = Unknown desc = failed to pull and unpack image "myserver.lan/docker-awx-ee:latest": failed to resolve reference "myserver.lan/docker-awx-ee:latest": failed to authorize: failed to fetch oauth token: unexpected status from POST request to https://myserver.lan/v2/token: 415 Unsupported Media Type
  Warning  Failed     11s   kubelet            Error: ErrImagePull
  Normal   BackOff    10s   kubelet            Back-off pulling image "myserver.lan/docker-awx-ee:latest"
  Warning  Failed     10s   kubelet            Error: ImagePullBackOff

I configure a container credentials for my awx-ee.

Thank you for your help

1 Like
2

Hello,

As a workaround i export my image from server registry. Copy it to my kubernetes server and import it with “k3s ctr images import” command.

Image is now available in my local repo on kub. I set it in AWX EE environment menu on IHM.

Did you have any idea why i can’t pull it from remote private registry ?

1 Like
3

@Gokusan31 hello, thanks for posting your question. To me the error says that there’s some kind of authentication issue. If i were you, i would also take a look at logs on the registry’s side.

4

Hello !

Thank you good idea.

On server side, there is a container registry on port 5030, and foreman which connect to this registry to make images availables thought port 443.

Logs on the registry:
nothing

Logs on foreman side match the error i get on AWX when pulling ee image:

2025-02-13T08:52:52 [I|app|570c7025] Started HEAD "/v2/test-docker-awx-ee/manifests/latest" for x.x.x.x at 2025-02-13 08:52:52 +0000
2025-02-13T08:52:52 [I|app|570c7025] Processing by Katello::Api::Registry::RegistryProxiesController#pull_manifest as HTML
2025-02-13T08:52:52 [I|app|570c7025]   Parameters: {"repository"=>"test-docker-awx-ee", "tag"=>"latest"}
2025-02-13T08:52:52 [I|app|570c7025]   Rendered api/v2/errors/unauthorized.json.rabl within api/v2/layouts/error_layout (Duration: 0.8ms | Allocations: 118)
2025-02-13T08:52:52 [I|app|570c7025]   Rendered layout api/v2/layouts/error_layout.json.erb (Duration: 1.3ms | Allocations: 212)
2025-02-13T08:52:52 [I|app|570c7025] Filter chain halted as :registry_authorize rendered or redirected
2025-02-13T08:52:52 [I|app|570c7025] Completed 401 Unauthorized in 130ms (Views: 2.5ms | ActiveRecord: 54.1ms | Allocations: 10443)
2025-02-13T08:52:52 [I|app|02487be0] Started POST "/v2/token" for x.x.x.x at 2025-02-13 08:52:52 +0000
2025-02-13T08:52:52 [I|app|02487be0] Processing by Katello::Api::Registry::RegistryProxiesController#token as HTML
2025-02-13T08:52:52 [I|app|02487be0]   Parameters: {"client_id"=>"containerd-client", "grant_type"=>"password", "password"=>"[FILTERED]", "scope"=>"repository:test-docker-awx-ee:pull repository:registry:pull,push", "service"=>"foreman.test.lan", "username"=>"admin"}
2025-02-13T08:52:52 [I|app|02487be0]   Rendered api/v2/errors/unsupported_media_type.json.rabl within api/v2/layouts/error_layout (Duration: 0.6ms | Allocations: 120)
2025-02-13T08:52:52 [I|app|02487be0]   Rendered layout api/v2/layouts/error_layout.json.erb (Duration: 1.1ms | Allocations: 214)
2025-02-13T08:52:52 [I|app|02487be0] Filter chain halted as :check_media_type rendered or redirected
2025-02-13T08:52:52 [I|app|02487be0] Completed 415 Unsupported Media Type in 6ms (Views: 2.4ms | ActiveRecord: 0.5ms | Allocations: 817)

Strange thing is when i try to pull manually with podman login than podman pull test-docker-awx-ee. It works without error message.

Looks like connection from awx as some differents parameters

5

@Gokusan31 do you use registry credentials? i think the registry connection parameters should be configurable

6

Yes i use the same credentials as the manual connection in command line.

Very weird