Documenting My Extension

Archives Ansible Project
1

I was cleaning up an rsync module I wrote months ago and didn’t document or really promote its existence.

https://github.com/tima/ansible-rsync

I think it’s in mostly in shape and was adding the docs to the module file (rsync) in there – though they haven’t been push to my github repo though. It occurred to me that what I’ve written is from the wrong perspective. Being in the rsync module file, I wrote docs explaining how that module work. It occurred to me that those docs should have been written them for the sychronize action that most users will use. Is that correct?

2

Correct, even some action plugins that have no remote components have documentation stubs, the idea is to document the interface of the action_plugin in that case, as the other will not be exposed.

In this case, I would think if they are paired together, they should share the same name (in your case, ‘synchronize’).

Interested in this, would like to see some examples of what it looks like in a playbook and what the capabilities are.

3

OK. That helps get my head straighter.

I was trying to develop this so that users had the options of using the module directly giving them additional (lower level) control of how Ansible uses rsync without having to deal with a raw command action. That was the intent. I need to dig thru the archives when we were discussing it on the list.

As for how the synchronize action works – its meant to be as simple and straight forward as possible (as opposed to a command action to rsync) by applying a lot of magic behind the scenes. In its most basic form:

synchronize: src=some/relative/path dest=/some/absolute/path

This will sychronize the path defined by dest on the inventory_hostname in context with the path defined by src on the localhost (by default – pass a delegate_to and that is used instead). By adding mode=pull you can reverse the sychronization.

synchronize: src=some/relative/path dest=/some/absolute/path mode=pull

As usual facts and variables are resolved so you could synchronize some path on multiple hosts to local subdirectories by hostname and the like. Relative paths are supported

The action detects being run using the local transport and takes appropriate action.

Two bonus features:

Provding a delete=yes argument will delete any files in the dest not found in the src using the --delete-after option in rsync. The default is to not delete files.

Providing a verbosity=1 argument turns on lots of information getting returned to Ansible. Increment verbosity to 2 you get even more information from processing. Take that to 3 and you get the firehose of information back from rsync. Good for lowel-level debugging if that helps you. The default is 0 (quiet).

The action always uses the archive mode, transfers with compression and delays updates until the end of the transfer to avoid leaving any hosts in some potentially broken, in-between state.

Some additional magic – the action gets the runner’s private key file for accessing the remote server and it uses ansible’s temporary files path in syncing. Also, if you work for a company like me that can’t do anything normal, you can specify an alternate location for rsync using a host facts, ansible_rsync_path.

Now to rework this in to the module documentation YAML and then do some testing.

4

Great, I think there is a lot of interest in a smoother rsync integration, so this is something i think we’d look to in core – and we can just document any limitations around keys and so on.

Let me know when you are ready and I’d be happy to help test.

5

A bit late to this, but I’ll add a “me too” - this would be great.

6

Yep, please send a pull request when you have something…

7

Awesome tip. That's way easier than the two step process I'm currently
using.

K

Kahlil (Kal) Hodgson GPG: C9A02289
Head of Technology (m) +61 (0) 4 2573 0382
DealMax Pty Ltd (w) +61 (0) 3 9008 5281

Suite 1415
401 Docklands Drive
Docklands VIC 3008 Australia

"All parts should go together without forcing. You must remember that
the parts you are reassembling were disassembled by you. Therefore,
if you can't get them together again, there must be a reason. By all
means, do not use a hammer." -- IBM maintenance manual, 1925

8

Just had a quick look at Tim’s module and it looks like simply setting the existing ‘rsync_path’ option to ‘sudo rsync’ would get around the sudo issue. I gather a simple patch to prepend ‘sudo’ to the rsync_path if ‘sudo’ is true at that point would work. I’ll see if I can get that working today.

K

9

After playing around with it a bit yesterday, I think you’ll want it to be ‘sudo -u <sudo_user> rsync’ if sudo_user is set.

e.g., this lets me sync things to the nagios user on my remote host without ownership ended up being set to my local user:

rsync -avz --rsync-path=‘sudo -u nagios rsync’ foo/ remotehost:~nagios/foo/

Make sense?

(why did I not know about this before…)

10

Ah yes, that does make sense. Only had a quick chance to look at this
yesterday before CVE-2013-2094 reared its ugly head.
At the moment I'm not sure how to get at the task context from within the
module code: I don't know how to tell if sudo is true or what the value for
sudo_user should be, or if this is even possible. Will have to do some
more research when I get a chance. Might just be easier to document the
workaround in the module.

K

Kahlil (Kal) Hodgson GPG: C9A02289
Head of Technology (m) +61 (0) 4 2573 0382
DealMax Pty Ltd (w) +61 (0) 3 9008 5281

Suite 1415
401 Docklands Drive
Docklands VIC 3008 Australia

"All parts should go together without forcing. You must remember that
the parts you are reassembling were disassembled by you. Therefore,
if you can't get them together again, there must be a reason. By all
means, do not use a hammer." -- IBM maintenance manual, 1925

11

I’ve think I’ve worked out all the kinks and got some proper docs completed.

https://github.com/tima/ansible-rsync

Take a look, try it out and let me know what you think. (That’s an invite to everyone, not just Michael.)

12

Glad to see this moving along.

Our process for getting stuff tested is to submit a pull request.

Then people can try out from there, and +1 if they think it works or comment on the pull request.

13

First I want to thank everyone that provided some feedback here and apologize I didn’t respond to it sooner. Unfortunately, I work for company that blocks access to Google Groups and have to do all my mailing list reading once I get home or from my iPhone. I’ve been heads down in my work that missed the rsync-path discussion until after I had sent my message out last Friday. Sorry if it seemed I was listening because it was true, but not intentional.

One thing I had done in my last commit was remove the rsync_path option because I didn’t think it was fully fleshed out enough and needed extra work. I didn’t think any needed it either. Obviously I was mistaken.

As it works now you can define a host variable ansible_rsync_path that the plugin will use to set the --rsync-path automatically. That path can also be set using an optional rsync_path in the sychronize task action call. Thinking about it more that seems off.

  • Should the host variable (ansible_rsync_path) override the task action argument? This seems like a big NO to me.

  • The rsync-path is for the remote server from the perspective of where rsync is being run. My plugin does not address where the rsync command is initialized. I’m just using ‘rsync’ and leaving things to the path.

This last point isn’t an issue for the “sudo” use being discussed, but for the unfortunate few like me it is useful for dealing with non-standard setups. We do have the means of defining an alternate path to sudo (ANSIBLE_SUDO_EXE) and the shell (ANSIBLE_EXECUTABLE) and even scripting engines via the ansible_*_interpreter host variables that I’m thinking I need to work additional logic in for rsync before release.

Thoughts?

14

OK I have submitted it here:

htttps://github.com/ansible/ansible/pull/2979