Sychronize module from Tim Appnel merged!

Archives Ansible Project
1

I know a lot of people were waiting for this one and wanted to pass along that this is now merged into 1.3 devel.

My testing of it was not comprehensive, I’m sure, so please have at.

Glad to see this one.

I haven’t pushed doc site rebuilds just yet, so here is the output from “ansible-doc synchronize”. I think you will all agree it is some really nice syntax.

[root@localhost ansible]# ansible-doc synchronize

SYNCHRONIZE

This is a wrapper around rsync. Of course you could just use the
command action to call rsync yourself, but you also have to add a
fair number of boilerplate options and host facts. You still may
need to call rsync directly via command' or shell’ depending on
your use case. The synchronize action is meant to do common things
with `rsync’ easily. It does not provide access to the full power of
rsync, but does make most invocations easier to follow.

Options (= is mandatory):

  • delete
    Delete files that don’t exist (after transfer, not before) in
    the `src’ path. (Choices: yes, no)

= dest
Path on the destination machine that will be synchronized from
the source; The path can be absolute or relative.

  • mode
    Specify the direction of the synchroniztion. In push mode the
    localhost or delgate is the source; In pull mode the remote
    host in context is the source. (Choices: push, pull)

  • rsync_path
    Specify the rsync command to run on the remote machine. See
    `–rsync-path’ on the rsync man page.

= src
Path on the source machine that will be synchronized to the
destination; The path can be absolute or relative.

  • verbosity
    An integer controlling the amount of information returned
    during processing. See the -v, --verbose' option of the rsync man page for details. If verbosity is not defined or a value of 0 is assumed, the –quiet’ option is passed and
    information is supressed.

Synchronization of src on the control machien to dest on the remote hosts

synchronize: src=some/relative/path dest=/some/absolute/path

Synchronization of two paths both on the control machine

local_action: synchronize src=some/relative/path dest=/some/absolute/path

Synchronization of src on the inventory host to the dest on the localhost in

pull mode
synchronize: mode=pull src=some/relative/path dest=/some/absolute/path

Synchronization of src on delegate host to dest on the current inventory host

synchronize: >
src=some/relative/path dest=/some/absolute/path
delegate_to: delegate.host

Synchronize and delete files in dest on the remote host that are not found in src of localhost.

synchronize: src=some/relative/path dest=/some/absolute/path delete=yes

Synchronize and return verbose information from the rsync transfer.

synchronize: src=some/relative/path dest=/some/absolute/path verbosity=1

Synchronize using an alternate rsync command

synchronize: src=some/relative/path dest=/some/absolute/path rsync_path=“sudo rsync”

2

Apparently I had merged this to a local branch. Now merged into the public devel branch.

If you were looking previously and were confused, this is why :slight_smile:

3

This module has been awhile in the making and took a couple of commits to support it’s function.

There are a couple of enhancements I thought of during this last push I want to work in. I didn’t want to make testing that module too much of a moving target.

The first is support of the rsync dry-run option thru Ansible’s check mode. Pretty straight forward really. Probably should have implemented it.

Implementation of the suppress MOTD option in rsync is another. That’s a fairly new option in rsync. I found at the day job we don’t have such a version that will support it and a very verbose MOTD that makes for some really noisy responses. rsync just dies if you use the option and it’s not supported. I thought of running rsync with the option and if it dies try again without it. If it dies the because of the MOTD option set a flag as an ansible_fact for next time. That seems a bit too hacky.

I thought of checking the version of rsync also but determining the version of rsync is pretty hacky also. They dump out a lot more than version when you ask it. So I’d have to parse that put of the response and set an ansible_fact for next time.

There was also a request to detect any changes rsync actually when run and reply ok or changed accordingly. Again rsync isn’t terribly helpful with integrating with an app like Ansible and would require greping for patterns in the response.

I don’t mind attempting these – I am not sure there isn’t a better way or if greping stdout is acceptable in the core. (BTW: I haven’t thought to check other modules to if and how they handle this.)

Thoughts?

4

Thanks for the follow up.

To answer that last question, searching stdout in Python is entirely fair game.

There’s a CLI function in module_common we use for most executions that will get you output back. More or less a wrapper around subprocess with some extra error checking:


    def run_command(self, args, check_rc=False, close_fds=False, executable=None, data=None, binary_data=False):
        '''

        Execute a command, returns rc, stdout, and stderr.
        args is the command to run

        If args is a list, the command will be run with shell=False.

        Otherwise, the command will be run with shell=True when args is a string.
        Other arguments:

        - check_rc (boolean)  Whether to call fail_json in case of

                              non zero RC.  Default is False.
        - close_fds (boolean) See documentation for subprocess.Popen().

                              Default is False.
        - executable (string) See documentation for subprocess.Popen().

                              Default is None.
        '''
5

I realise this module is still in a state of flux, and currently out devel branch.

Anyhow, i’ve been working on it here as i do need it, still the other issues remain, but using local_action, I do have the change detection working.

I found the simplest way was to turn --quiet off and pass in an --out-format= flag to rsync. I used a pattern such as --out-format=‘%i %n%L’ which makes rsync return a list of every modification, prefixed by . Then using run_command, i simply check for the existance of and return if changed or not.

Of course could be anything, but using the out-format seemed most convenient rather than trying to decipher the default -i flag format.

I will test this further, and once the synchorinize module is back in the branch i will create a pull request.

6

It seemed to have other issues about not running locally when it needed to as well.

Also I’m not positive you do need it, because you can still shell out to rsync? Can you elaborate on usage?

I agree it would be very nice to have this abstraction here though, and I still want this.