I’m trying to set up a fresh AWX instance and I’ve run into an issue while creating/syncing a project.
I’m trying to pull a private repo from GitHub using an SSH credential, but no matter what I do, the sync job always hangs on the git project update task, with the message “The authenticity of host ‘github.com (140.82.121.3)’ can’t be established.”.
Seems like GitHubs SSH host key is not being accepted for some reason?
Below are the project settings. I’ve also tried a different URL syntax (ssh://github.com/username/project) but that didn’t do me any good either. Do I need to somehow inject the GitHub host key or should AWX just handle that itself? I don’t remember this ever being necessary on my previous AWX install. Perhaps I’m just missing something?
This seems to be related to some recent change in ansible/AWX-EE. I just manually created an Control Plane EE with Ansible-Runner pinned to stable-2.11 and configured my deployment to use it. The issue has now gone away, along with the warning about running a development version of Ansible.
Thanks for posting - that is very interesting because I am getting that trying to get a project to sync for a customer today.
When I set it up identically on my environment it works fine, but in the customer’s environment I get the message about being unable to verify the identity of gitlab.com (in my case I’m using Gitlab).
I will try your solution tomorrow for the customer and see if that helps. Is this how you have configured your EE?
Glad to hear I’m not the only one experiencing this, seems like it might a recent breakage in ansible-devel or something along those lines?
Either way, something is breaking in the Control Plane EE, and while my solution is far from ideal, at least I can sync projects again:
As far as I know, this should force the AWX Control Plane stuff (like project updates) to run with a fixed, older Ansible version. I don’t think this is at all indented by the AWX team, but oh well.
Build that EE image and push it to some registry/upload it to your k8s cluster
Adjust your AWX deployments spec and add your custom Control Plane EE like so:
Again, I don’t think this is a proper solution, just a hack to get up and running. But it’s better than nothing. I also don’t think it’s possible to achieve this via the UI, you need to change the operator deployment definition (see: https://github.com/ansible/awx-ee/issues/72)
Thanks for taking the time to document your solution.
I managed to fix it in a different way. I realised that the container was not trusting gitlab.com and maybe it was asking a question about whether to trust it but that was not being displayed in AWX.
So I started a shell and tried to clone the repo there, and sure enough, it asked whether I wanted to trust it. I said yes. The clone added the gitlab.com host to the list of known hosts, then failed because I’m not using the right SSH key at the bash shell. However since then, it works in AWX!
Here’s the output from my session:
USER@ip-172-29-247-100:~$ kubectl get pods
NAME READY STATUS RESTARTS AGE
awx-operator-controller-manager-66d6589d78-r5hwc 2/2 Running 0 25h
awx-postgres-0 1/1 Running 0 25h
awx-76dc548cb-jfhts 4/4 Running 0 25h
USER@ip-172-29-247-100:~$ kubectl exec -it awx-76dc548cb-jfhts /bin/bash -c awx-ee
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] – [COMMAND] instead.
bash-4.4$ pwd
/runner
bash-4.4$ git clone git@gitlab.com:USER/REPO.git
Cloning into ‘REPO’…
The authenticity of host ‘gitlab.com (172.65.251.78)’ can’t be established.
ECDSA key fingerprint is SHA256:HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added ‘gitlab.com,172.65.251.78’ (ECDSA) to the list of known hosts. git@gitlab.com: Permission denied (publickey,keyboard-interactive).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
bash-4.4$
