synchronization module take 3

Archives Ansible Project
1

I’ve submitted a pull request for the synchronization module I’ve been working on for anyone to review.

https://github.com/ansible/ansible/pull/4167

There is quite a bit different since the last go. A lot of the changes in this go are thanks to @smoothify. Most importantly, he figured out the problem that lead to this module getting yanked from 1.3.

  • The --dry-run option is included in the call to rsync if Ansible is being run with the --check flag.
  • Added individual switches for each individual option --archive wraps. i.e --links, --recursive, --perms
  • The rsync --archive option can be turned off to support the individual options above. It’s still on by default.
  • Change detection – thanks to an ingenious trick @smoothify figured out the module more accurately detects and reports if the underlying rsync module made any changes.
  • Proper sudo mode handling.

While a number of new options have been added, I really worked on making the number of options an action must define to perform a sync operations to a minimum. In my experience most people invoke rsync with the --archive option, but there are times when you don’t want to or can’t use one or two options that are enabled. Rather than turn off the archive mode and individually set each option like this one where I can’t modify timestamps.

sychronize: src=/path/to/src dest=/path/to/dest archive=no recursive=yes links=yes perms=yes group=yes owner=yes

I turn off archive and set all the options that it would have enabled except for times. I went a different direction where you can “subtract” options that the archive option enabled like this.

sychronize: src=/path/to/src dest=/path/to/dest times=no

If all you want is a recursive without changing times or setting ownwer, permissions etc you can do this:

sychronize: src=/path/to/src dest=/path/to/dest archive=no recursive=yes

One other thing I think is worth highlighting is change detection and that I had to remove the verbosity controls. Changes are detected by configuring the output format to include a marker. If that marker is seen in the out we can assume some type of change happened. What I found in testing though is the --quiet suppresses that output (no duh) and verbosity output (like -vv or -vvv) lists every file with the changed marker even when a change has not occurred. SO I removed it for now to ship. We can revisit if the ends up being a big problem.

Give it a try and let me know if you find any issues.

2

This sounds really great. Thanks to you and @smoothify for your continued efforts here. It’s going to help a lot of folks.

3

thanks for your perseverance on this! I know its not an easy one to get
right.

I'll be testing it soon and will let you know if there are any issues.

4

  • it seems the documentation is out of sync with the module, mode= is not supported.
    “msg: unsupported parameter for module: mode”

  • the default ‘push’ mode operation seems to not work, default seems to be src is in inventory_hostname, not the local host (unless local action or delegate_to are set). dest seems to be same machine it executes on (inventory_hostname, or local/delegated machine).

will keep testing.

5

What’s the rationale for using the module name “synchronize” instead of “rsync”?

James

6

James,

Same reason we don’t call the copy module “scp_or_sftp_according_to_configuration_option” :slight_smile:

7

Brian: I can’t reproduce either of these issues. Are you sure you pulled both runner/action_module/synchronize.py and library/files/synchronize? mode is actually handled in the action module and is discarded before what gets sent out to the remote server. if you were to copy just what’s under library I’d expect you’d see these errors.

8

ah, missing runner .... ignore previous posts.

9

And yet we have apt and yum modules. Just trying to understand the logic behind the different approaches.

James