Make it possible to nest connection plugins

Archives Ansible Project
1

Hi all,

I’m getting my feet wet in ansible and I’m pretty exited so far. But I think I discovered a use case, which might be of interest for others:

We have numerous hosts with jails on them and would like to manage the jails with ansible - and not all jails are accessible via SSH, but all hosts are accessible via SSH. I would like to be able to “nest” the jail connection plugin with the ssh connection plugin, so I can connect to a host normally and then connect via the jail plugin to a specific jail and execute commands in there.

Of course I would have to specify on which host to look for the jail.

What do you think about this? If its not too hard to implement, I would consider writing a patch, but would need some help with that.

If thats already possible, please ignore this - I was asking on the IRC-Channel about this and all I found out was, that a jail connection plugin already exists, but is limited to local use.

Cheerio,
Filias

2

It’s not going to be possible to nest them.

What I think you are asking for is some way to do “remote jail management” which is going to be, unfortunately, probably a new connection plugin that might not be easy.

I hate to say it, but this is why cloud VMs and things like bridged networking are so great, where each VM can get an IP.

I’m open to the idea of seeing another connection plugin for this, but I doubt we’d want to maintain it in core. I could be wrong - I can see the idea of having something like this to be a pretty cool thing for some very specific use cases.

3

Hi

Hi all,

I'm getting my feet wet in ansible and I'm pretty exited so far. But I think I
discovered a use case, which might be of interest for others:

We have numerous hosts with jails on them and would like to manage the jails
with ansible - and not all jails are accessible via SSH, but all hosts are
accessible via SSH.

It sounds more like you need to tweak your SSH config for these jailed
hosts - e.g. a fragment like this in $HOME/.ssh/config:

  Host somejailedhost
     ProxyCommand ssh jailkeeperhost nc %h %p

This should allow you to use "ssh" (and scp for that matter) to
interact directly with the jailed host. And Ansible should be able to
take advantage of that.

Hope this helps

4

I thought the more generic “nesting” would be a good thing, because the next guy might want to control Docker instances or Containers on a remote host without them having a public IP. Maybe I’ll try to write a connection plugin.

@Karl: This is not possible, because some of the Jails don’t even run SSH. You SSH into the Host and run commands inside the jails via “jexec”.

5

My suspicions are that's going to be technically possible with how
connection plugins work (or even in general, very very complicated). SSH
jump hosts would probably be a reasonable thing to do.

6

The way connection plugins work, it’s not going to be possible to nest existing ones. But what you can do is, create a new API for “connection plugin layers” that can be put on top of an existing “base” connection plugin. Something like: class ConnectionLayer(object): def init(self, target_address): “”“target_address is a part of ansible_ssh_host for this plugin, like jail’s path”“” def wrap_command(self, command): “”“Wraps the command in chroot/jail/etc.”“” def unwrap_result(self, rc, stdout, stderr): “”“Takes whatever the wrapped command returned after being run by the base connection plugin, and returns a new tuple of (rc, stdout, stderr)”“” def wrap_path(self, path): “”“Takes a path inside container/chroot/jail and translates it into host path.”“” I’m not really sure if passing continuations or some “lower layers” object wouldn’t make this a better API, but I think it touched the most important point. I would actually like to see something like this, as I’ve written a OpenVZ-over-SSH connection plugin at work (and I can already give you a hint: don’t try to put host address anywhere but ansible_ssh_host, this variable gets special treatment. I’ve tried to have separate ansible_vzenter_ctid, and in the end had to rewrite that part).

7

Yeah I’m suspecting this will be really hard to debug, if it works at all.

For instance, theoretically, you should be able to nest anything in anything.

So instead of using SSH jumphosts, maybe you SSH into a SSH. Well, to do that, you have to move a file to a temporary location on an interemediate host. (This is a bad example, because jumphosts are way better in this case).

I think it only fits for the “remote jail management cloud” case, which, while I’ll say it’s interesting, I fear it’s too far off the beaten path when most people are going to be moving to a world where the virtual nodes have valid IP addresses.

In any case where this is possible, those nodes could just be surfaced by an inventory script that understands what’s out there.

I guess what I’m saying is I’m rather skeptical about the idea of having a wrap system, I could see an implementation and be convinced, but I’m also not sure this is important enough to maintain an architectural layer for it.

Sorry, I don’t mean to be crushing a cool idea - it’s just going to take some selling and I’m not there yet.

8

oh, i forgot the links to the nsenter stuff... here they are:

- http://blog.docker.com/2014/06/why-you-dont-need-to-run-sshd-in-docker/
- someone proposed a local docker connection here
   https://github.com/ansible/ansible/pull/7971

cheers,

Alberto

9

Hi,

sorry for the confusing precendence of my latter message, but seems that
this message had some problems with gmane, so i'm resending it....

    >> I thought the more generic "nesting" would be a good thing, because
    >> the next guy might want to control Docker instances or Containers on
    >> a remote host without them having a public IP. Maybe I'll try to
    >> write a connection plugin.
    > The way connection plugins work, it's not going to be possible to nest
    > existing ones.
    > But what you can do is, create a new API for "connection plugin
    > layers" that can be put on top of an existing "base" connection
    > plugin.
    > Something like:

    > class ConnectionLayer(object):
    > def __init__(self, target_address):
    > """target_address is a part of ansible_ssh_host for this
    > plugin, like jail's path"""

    > def wrap_command(self, command):
    > """Wraps the command in chroot/jail/etc."""

    > def unwrap_result(self, rc, stdout, stderr):
    > """Takes whatever the wrapped command returned after being run
    > by the base connection plugin,
    > and returns a new tuple of (rc, stdout, stderr)"""

    > def wrap_path(self, path):
    > """Takes a path inside container/chroot/jail and translates it
    > into host path."""

any update on this? I would like to use nsenter [1]_ to run commands
inside the container (how to transfer files is a detail i have yet to
analyze) to work on docker containers either locally or on a remote
host, and i would really like to contribute to and use a layer like
this. Unfortunately i'm quite new to ansible and not all the details
involving connections are clear to me.
    
    > I would actually like to see something like this, as I've written a
    > OpenVZ-over-SSH connection plugin at work (and I can already give you
    > a hint: don't try to put host address anywhere but ansible_ssh_host,
    > this variable gets special treatment. I've tried to have separate
    > ansible_vzenter_ctid, and in the end had to rewrite that part).

Nice! openvz is the other virtualization system i use :slight_smile:

So, where you store the ctid? in to so other host related variable? Can
you say more about having something like a variable ansible_vzenter_ctid
is not feasible?

Cheers,

Alberto

10

Hi,

sorry for the confusing precendence of my latter message, but seems that
this message had some problems with gmane, so i'm resending it....

     >> I thought the more generic "nesting" would be a good thing, because
     >> the next guy might want to control Docker instances or Containers on
     >> a remote host without them having a public IP. Maybe I'll try to
     >> write a connection plugin.
     > The way connection plugins work, it's not going to be possible to nest
     > existing ones.
     > But what you can do is, create a new API for "connection plugin
     > layers" that can be put on top of an existing "base" connection
     > plugin.
     > Something like:

     > class ConnectionLayer(object):
     > def __init__(self, target_address):
     > """target_address is a part of ansible_ssh_host for this
     > plugin, like jail's path"""

     > def wrap_command(self, command):
     > """Wraps the command in chroot/jail/etc."""

     > def unwrap_result(self, rc, stdout, stderr):
     > """Takes whatever the wrapped command returned after being run
     > by the base connection plugin,
     > and returns a new tuple of (rc, stdout, stderr)"""

     > def wrap_path(self, path):
     > """Takes a path inside container/chroot/jail and translates it
     > into host path."""

any update on this? I would like to use nsenter [1]_ to run commands
inside the container (how to transfer files is a detail i have yet to
analyze) to work on docker containers either locally or on a remote
host, and i would really like to contribute to and use a layer like
this. Unfortunately i'm quite new to ansible and not all the details
involving connections are clear to me.

I'll see what I can do, but I wouldn't hold my breath about opening the one I wrote at work.
And hey, it's always a good time to rewrite something :slight_smile:

        > I would actually like to see something like this, as I've written a
     > OpenVZ-over-SSH connection plugin at work (and I can already give you
     > a hint: don't try to put host address anywhere but ansible_ssh_host,
     > this variable gets special treatment. I've tried to have separate
     > ansible_vzenter_ctid, and in the end had to rewrite that part).

Nice! openvz is the other virtualization system i use :slight_smile:

So, where you store the ctid? in to so other host related variable? Can
you say more about having something like a variable ansible_vzenter_ctid
is not feasible?

I'm putting it in ansible_ssh_host like this:
some_container ansible_connection=vzenter ansible_ssh_host=<SSH host>|<CTID>

Using variable other than ansible_ssh_host is not feasible, as they are not provided to connection plugins.
You can extract them from runner (as connection plugins do get access to runner), but you can't support the delegate_to: '{{ item }}' pattern this way (as you don't get access to loop items).

11

“I’m putting it in ansible_ssh_host like this:
some_container ansible_connection=vzenter ansible_ssh_host=|”

Maybe like this:

ansible_ssh_host=hostname;container=CTID

If that could be implemented natively in Ansible’s SSH.py, without doing too much damage, I would be SUPER interested.

I’d like to see something somewhat pluggable, and in order to take it, it would need to support Docker initially.

Though I do feel the “no SSH in containers” statement is a little religious. They want to encourage automation, but when the automation uses SSH, that’s totally cool :slight_smile:

12

I like that syntax, thanks. I’m thinking about doing something like: ansible_connection=stack ansible_ssh_host==;=<“hostname” for that layer> I don’t think ssh.py (or any other specific connection plugin) is a good place for this, I think it should rather be done as separate plugin or in core. That’s because I’d like to see the same layers used with local/ssh/accelerate. Right, as Ansible already has a plugin system I think that should be doable :slight_smile: And being pluggable is a big part of it for me - I don’t want to have separate plugin for each base plugin/layer combination. For me it was either jump hosts or entering containers, and the latter seemed more robust (I can access containers with broken networking :-))

13

“I don’t think ssh.py (or any other specific connection plugin) is a good place for this, I think it should rather be done as separate plugin or in core.
That’s because I’d like to see the same layers used with local/ssh/accelerate.”

I’m not sure that’s a good idea.

(A) all distros are moving to new enough SSH that paramiko is going to be less important, and with no paramiko there will be no need for accelerate
(B) the concept of “nesting” here is probably going to be way more difficult than just doing it for one plugin.
(C) yes, the local case would require ssh, but this seems less important in the long run.

14

Yes, it is. I’ve started hacking on it (see . It only supports chroot, but jail shouldn’t be hard to add), and I’ve stumbled upon some problems: * to have a sudo layer as robust as sudo functionality in SSH, layers would need some way to get partial output from commands. I’m thinking about using generators or something similar, but considering that “yield from” and mixing yield with return were only added somewhere in py3, it might turn out pretty ugly. * put/fetch might need some more freedom than just changing the path on managed host, but that shouldn’t be hard to add (except maybe for risking a loop between exec_command and put/fetch).

15

Hi all,

Yes, it is. I’ve started hacking on it (see https://github.com/ktosiek/ansible-nested-connections. It only supports chroot, but jail shouldn’t be hard to add), and I’ve stumbled upon some problems:

I was wondering what the status was on this one? I also need the configure jails on ssh-accessible jailhost functionality. If there is no further development I might try to add support for jails for this.

Cheers
Paul

"I don’t think ssh.py (or any other specific connection plugin) is a good place for this, I think it should rather be done as separate plugin or in core.

That’s because I’d like to see the same layers used with local/ssh/accelerate."

I’m not sure that’s a good idea.

(A) all distros are moving to new enough SSH that paramiko is going to be less important, and with no paramiko there will be no need for accelerate

(B) the concept of “nesting” here is probably going to be way more difficult than just doing it for one plugin.

Yes, it is. I’ve started hacking on it (see https://github.com/ktosiek/ansible-nested-connections. It only supports chroot, but jail shouldn’t be hard to add), and I’ve stumbled upon some problems:

  • to have a sudo layer as robust as sudo functionality in SSH, layers would need some way to get partial output from commands.
    I’m thinking about using generators or something similar, but considering that “yield from” and mixing yield with return were only added somewhere in py3, it might turn out pretty ugly.
  • put/fetch might need some more freedom than just changing the path on managed host, but that shouldn’t be hard to add (except maybe for risking a loop between exec_command and put/fetch).

(C) yes, the local case would require ssh, but this seems less important in the long run.

"I’m putting it in ansible_ssh_host like
this:

some_container ansible_connection=vzenter
ansible_ssh_host=|"

Maybe like this:

ansible_ssh_host=hostname;container=CTID

I like that syntax, thanks. I’m thinking about doing something like:

ansible_connection=stack ansible_ssh_host=<connection

=;<connection
=<“hostname” for that layer>

If that could be implemented natively in Ansible’s SSH.py,
without doing too much damage, I would be SUPER interested.

I don’t think ssh.py (or any other specific connection plugin) is a
good place for this, I think it should rather be done as separate
plugin or in core.

That’s because I’d like to see the same layers used with
local/ssh/accelerate.

I’d like to see something somewhat pluggable, and in order
to take it, it would need to support Docker initially.

Right, as Ansible already has a plugin system I think that should be
doable :slight_smile:

And being pluggable is a big part of it for me - I don’t want to
have separate plugin for each base plugin/layer combination.

Though I do feel the “no SSH in containers” statement is a
little religious. They want to encourage automation, but when
the automation uses SSH, that’s totally cool :slight_smile:

For me it was either jump hosts or entering containers, and the
latter seemed more robust (I can access containers with broken
networking :-))

Hi,

sorry for the confusing precendence of my latter
message, but seems that

this message had some problems with gmane, so i’m
resending it…

“Tomasz” == Tomasz Kontusz tomasz....@gmail.com
writes:

I thought the more generic “nesting”
would be a good thing, because

the next guy might want to control
Docker instances or Containers on

a remote host without them having a
public IP. Maybe I’ll try to

write a connection plugin.

The way connection plugins work, it’s
not going to be possible to nest

existing ones.

But what you can do is, create a new
API for "connection plugin

layers" that can be put on top of an
existing “base” connection

plugin.

Something like:

class ConnectionLayer(object):

def init(self,
target_address):

“”"target_address is a part of
ansible_ssh_host for this

plugin, like jail’s path"“”

def wrap_command(self, command):

“”“Wraps the command in
chroot/jail/etc.”“”

def unwrap_result(self, rc,
stdout, stderr):

“”"Takes whatever the wrapped
command returned after being run

by the base connection plugin,

and returns a new tuple of
(rc, stdout, stderr)“”"

def wrap_path(self, path):

“”"Takes a path inside
container/chroot/jail and translates it

into host path.“”"

any update on this? I would like to use nsenter [1]_
to run commands

inside the container (how to transfer files is a
detail i have yet to

analyze) to work on docker containers either locally
or on a remote

host, and i would really like to contribute to and use
a layer like

this. Unfortunately i’m quite new to ansible and not
all the details

involving connections are clear to me.

I’ll see what I can do, but I wouldn’t hold my breath about
opening the one I wrote at work.

And hey, it’s always a good time to rewrite something :slight_smile:

I would actually like to see
something like this, as I’ve written a

OpenVZ-over-SSH connection plugin at
work (and I can already give you

a hint: don’t try to put host address
anywhere but ansible_ssh_host,

this variable gets special treatment.
I’ve tried to have separate

ansible_vzenter_ctid, and in the end had
to rewrite that part).

Nice! openvz is the other virtualization system i use
:slight_smile:

So, where you store the ctid? in to so other host
related variable? Can

you say more about having something like a variable
ansible_vzenter_ctid

is not feasible?

I’m putting it in ansible_ssh_host like this:

some_container ansible_connection=vzenter
ansible_ssh_host=|

Using variable other than ansible_ssh_host is not feasible,
as they are not provided to connection plugins.

You can extract them from runner (as connection plugins do
get access to runner), but you can’t support the
delegate_to: ‘{{ item }}’ pattern this way (as you don’t get
access to loop items).

Cheers,

Alberto

–

You received this message because you are subscribed to
the Google Groups “Ansible Project” group.

To unsubscribe from this group and stop receiving emails
from it, send an email to ansible-proje...@googlegroups.com.

To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/53E3DA41.7050904%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

–

You received this message because you are subscribed to the Google
Groups “Ansible Project” group.

To unsubscribe from this group and stop receiving emails from it,
send an email to ansible-proje...@googlegroups.com.

To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgwx54dt5OAZOyU4ZG28JeTwgvKw6cffGAUtAWiScbZiuw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

–

You received this message because you are subscribed to the Google Groups “Ansible Project” group.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.

To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/53E3F519.2030501%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

–

You received this message because you are subscribed to the Google Groups “Ansible Project” group.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.

To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgxPYib1LYxQbvoz8daoFhB6hWgtuzS_SajAS9q%3Dz0rwhA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

are running
I was wondering what the status was on this one? I also need the configure jails on ssh-accessible jailhost functionality (I’m a bit surprised this is not considered a common use case TBH).

16

No plans for this at this time.

I understand the appealing nature of this one, I think it might make sense as a non-core connection plugin perhaps.

But the usage of FreeBSD jails is small now, and I don’t think we’d want to continue to maintain this as sort of a “cloud starter kit”.

Hope that makes sense.

17

Dear Michael,

OK, thanks for the heads up. I’ll see if I get this working myself then.

cheers
P