Git Module Repo Scheme Change?

Archives Ansible Project
1

I’m updating a project (setting up a disposable dev environment) from ansible 1.6.2 to 1.7. (Installed and upgraded using pip into a venv).

I’ve been setting up a user on the host that I’m configuring, with a full set of files in .ssh for things like the config (to assign an alias to github), known_hosts, and my key to connect to github.

I have a play that used to look like:

  • name: Clone repositories
    git: force=no
    repo=ssh://gh/me/{{ item }}.git
    dest=/home/foo/projects={{ item }}
    key_file=/home/foo/.ssh/github_key
    recursive=yes
    update=yes
    accept_hostkey=yes
    with_items:
  • bar
  • baz
  • quux

After the upgrade, I started getting this error from each repo:

failed: [host] => (item=…) => {“failed”: true, “item”: “…”}
msg: failed to add gh hostkey: getaddrinfo gh: No address associated with hostname

Getting rid of accept_hostkey=yes switched the error message to:

msg: gh has an unknown hostkey. Set accept_hostkey to True or manually add the hostkey prior to running the git module

I did notice some messages about adding 192.30.252.128 (along with .130) to my list of known_hosts when
I ssh’d in or cloned the repo manually. (That doesn’t seem relevant, but the address looked suspicious to me).

The host that I’m configuring is running ubuntu trusty, inside an lxc.

Changing the repo address to ssh://git@github.com/me/{{ item }}.git seemed to have fixed the problem at first glance.

But now I can’t actually connect to any of those projects that I just cloned. “Permission denied (publickey).”

I can go through and run “git set-url” on each repo to go back to using what I think I actually want, but that’s a huge part of the pain I use ansible to avoid.

Have I missed something about this over the past few months? Or in the docs? Is this intended behavior and I’m missing something obvious?

Thanks, and apologies in advance if this is noise,
James

2

Can you ping “gh” from that host and is it available in DNS?

3

Sorry, I wasn't as clear as I thought/wanted. ~/.ssh/config has a host
entry that sets up an alias from gh to github.com.

After the playbook has run, I can go through the repositories that
were cloned as ssh://git@github.com/me/{{item}} and run "git remote
set-url ssh://gh/me/{{item}}" in each cloned repo to get things
configured the way I think I want. (I can unsuccessfully ssh and
pull/push without a hitch).

I thought this might be the key to my problem: I'm using a static
hosts file, with 2 groups. This host is the only one in both groups.
One group uses a sudoer (for configuring the big-picture OS parts),
the other a regular user that I want to use for day-to-day work. I'd
been using the playbooks to control which user does what, based on the
hosts associated with the playbook. For the first step this time
around, I had to comment out the non-privileged group to keep it from
overriding the user settings from the privileged group.

Running with -vvvv verified that ansible is connecting as the normal
user I want for this play (so that user's ~/.ssh/config should be in
effect...shouldn't it?)

Completely commenting the "privileged" group out of hosts didn't make
any difference that I can see.

Just for the sake of completeness, my hosts file looks like:

[sudoer]
#10.0.3.152 ansible_ssh_user=special ansible_ssh_pass=1234
ansible_sudo_pass=1234

[normal]
10.0.3.152 ansible_ssh_user=normal ansible_ssh_pass=5678

Just for grins, I added a play directly above the one for cloning the repos:
- name: Check ssh
  remote_user: normal
  command: ssh gh

It failed (as expected) with:
"stderr: PTY allocation request failed on channel 0
Hi me! You've successfully authenticated, but GitHub does not provide
shell access.
Connection to github.com closed."

Which seems...really weird to me. It doesn't looks like the git module
is doing anything except building up a command line to basically do
this.

That led me to try some other command-based plays to clone repos
without looping:

- name: https clone
  remote_user: normal
  command: git clone https://github.com/me/foo.git
* [worked]

- name: alt ssh clone
  remote_user: normal
  command: git clone ssh://git@github.com/me/baz.git
* [Permission denied (publickey).]

- name: check ssh
  remote_user: normal
  command: git clone ssh://gh/me/quux.git
* [worked]

Which leaves me more confused. How is the git module managing to clone
a url that my "alt ssh clone" refused?

On nights like this, when I've been staring at all the tickets about
github and ssh that might possibly be related, I wonder if the people
at ansible ever curse at the whim that led them to decide to support a
monstrosity like this.

Since, after all, I am *not* the target market.

Don't get me wrong. I appreciate how much easier you've made my life.
I'm just having flashbacks to all those bugs that refuse to admit that
I fixed them. And feeling bad because I don't know your code base well
enough yet to have fixed this myself.

Thank you,
James

4

It’s likely the accept_hostkey magic only is firing on the initial clone, right now the git module is not so good about changing the repo to a different repo, but works fine when changing the branch/tag/etc.

This is more or less - https://github.com/ansible/ansible/issues/4658 - if you’d like to comment on the ticket so we also think about hostkeys when we get to this item that would be great.

5

It's likely the accept_hostkey magic only is firing on the initial clone,

That sounds like it may definitely be related. And I know all too well
how frustrating it can be to have people who don't understand the
issue drone on about "it seems to me that it looks like x is
happening" when I know very well that it's actually y (and probably
z).

Can you point me to a good place (including code) to look at what that
"accept_hostkey magic" actually does? I was under the impression that
it's really about having the host [being modified via ansible] accept
the identity of a host on the other end of a git/ssh connection it
hasn't been manually told to trust.

right now the git module is *not* so good about changing the repo to a
different repo, but works fine when changing the branch/tag/etc.

I seem to be breaking all the rules for asking questions well.

My work-around is to manually change the repo's url after I have
ansible put the basic pieces in place. Ansible seems to clone
ssh://git@github.com/me/{{ item }}.git fine. It fails on
ssh://gh/me/{{ item }}.git, which is what I need to actually use the
repo.

It isn't a horrible work-around, since I'm only dealing with 11
repositories, but it will get annoying when I start coping with
distributed scenarios that involve 5 or 6 hosts.

This is more or less - https://github.com/ansible/ansible/issues/4658 - if
you'd like to comment on the ticket so we also think about hostkeys when we
get to this item that would be great.

I'll be happy to do this (I *like* contributing, even if it's just
notes on a bug report), but I wanted to double-check first. I don't
want my poor writing skills to muddy the water.

From a Clueless Outsider's perspective, that bug looks like a *very*

distant relative.

I'm trying to clone a git repo in ansible 1.7 using a uri pattern that
worked fine in 1.6.2. [I'm pretty sure I'm using exactly the same
pattern in 1.6.10...but I'll need to check in at the office tomorrow].
That pattern centers around an ssh alias I set up in ~/.ssh/config [on
the host]. When I switch to a "more traditional" uri pattern, ansible
clones the repo without a problem, but then I can't actually use it,
because I don't have any map inside that VM from the repo url to the
private ssh key that has rights to the repo.

Maybe that's the key (I didn't intend the pun, honest). I'm not using
ssh_agent. Because, up to now, figuring out how to do without it has
always seemed easier than figuring out how to make it work with our
build system. Which means I forgot that it even exists about 6 months
ago.

Or maybe that's totally unrelated. I've watched QA guys go off on that
same sort of tangent more times than I can count over the years. If
you want me to add this to the "clone repo a second time over the
first" bug, I'll believe you and do it happily. But I *did* want to
make sure I'm filing it in the correct spot.

Thank you *so* much for such an amazing tool,
James

6

" I was under the impression that
it’s really about having the host [being modified via ansible] accept
the identity of a host on the other end of a git/ssh connection it
hasn’t been manually told to trust."

The usage of accept_hostkey=yes is an indication that you are telling it manually to trust. Basically it will call (on the module side) the right commands to add the fingerprint of the remote server to known_hosts, such that you don’t have to do manually manage or add it to known hosts. Obviously, in doing so, you must trust the remote entity, so not everyone may be comfortable with doing this at runtime.

“From a Clueless Outsider’s perspective, that bug looks like a very
distant relative.”

Nope, it’s the same thing. If you specify a new origin repo to the git module, it needs to not only blow away the current content to reclone, but also consider accepting a new hostkey.

7

The usage of accept_hostkey=yes is an indication that you are telling it
manually to trust. Basically it will call (on the module side) the right
commands to add the fingerprint of the remote server to known_hosts, such
that you don't have to do manually manage or add it to known hosts.
Obviously, in doing so, you must trust the remote entity, so not everyone
may be comfortable with doing this at runtime.

Thank you. That's what I thought.

"From a Clueless Outsider's perspective, that bug looks like a *very*
distant relative."

Nope, it's the same thing. If you specify a new origin repo to the git
module, it needs to not only blow away the current content to reclone, but
also consider accepting a new hostkey.

I'm not doing that. Well, not through ansible.

It will not clone ssh://gh/me/{{ item }}.git. So I had it clone
ssh://git@github.com/me/{{ item }}.git instead.

That works fine, except that the user on the freshly-configured host
doesn't have any idea which ssh key to use for that url. So, after
ansible's out of the picture, I've manually run set-url to let me do
things that are part of my normal old-fashioned work flow, like git
pull and push.

8

“That works fine, except that the user on the freshly-configured host
doesn’t have any idea which ssh key to use for that url.”

I see. That sounds like it might still be related to the set-url being a host that doesn’t resolve?