Strange issue with the copy speed of a 1.5G large file.

Archives Ansible Project
1

I am using the following snippet to execute the playbook, this is the only way I can solve the deployment of servers for the first time as the inventory is not present at that time and lot of orchestration is needed to deploy the cluster with several nodes.

def run_playbook(listVMs, playbook, user, extra_vars=None, key_file=None):
print(datetime.datetime.now())
stats = callbacks.AggregateStats()
playbook_cb = callbacks.PlaybookCallbacks(verbose=utils.VERBOSITY)
runner_cb = callbacks.PlaybookRunnerCallbacks(stats, verbose=utils.VERBOSITY)

pb = ansible.playbook.PlayBook(
host_list=listVMs,
playbook=playbook,
forks=20,
remote_user=user,
remote_pass=‘xxxxx’,
runner_callbacks=runner_cb,
callbacks=playbook_cb,
stats=stats,
extra_vars=extra_vars
)

pb.run()

running one of the playbook with the copying the large 1.5G file is taking excruciatingly 10 minutes.

  • name: copy large file
    copy: src={{ xxx_largefile }} dest=/yyyy owner=zzz group=zzz

If I run the same playbook from the command line, it is very quick less than a minute.
ansible-playbook --inventory-file=host.ini playbookcopy.yml

What might be the difference, I have the latest ansible installed…

How can I make it easy using the above the code snippet.

2

I swapped copy module to synchronize and it worked like charm.
But , it is still puzzling me why copy module behaves differently in those 2 scenarios.

3

“I am using the following snippet to execute the playbook, this is the only way I can solve the deployment of servers for the first time as the inventory is not present at that time and lot of orchestration is needed to deploy the cluster with several nodes.”

an external inventory script seems appropriate for this and will avoid the problem of your trying to use the API directly.

“running one of the playbook with the copying the large 1.5G file is taking excruciatingly 10 minutes.”

This is effectively using scp or sftp, but it sounds like you may be using a different connection type or something due to the way you have invoked ansible, or maybe aren’t loading the configuration.

Hard to say.

Regardless, this is really an API question, so a bit more suited for ansible-devel. More of an FYI for the future.

The primary purpose of the API is to support /usr/bin/ansible-*, so I can’t really provide a lot of debugging help beyond the above.

We do support the REST API in Tower to a much greater extent, though I’d possibly suggest looking through /usr/bin/ansible and ansible-playbook and seeing how they construct things, with an expectation this is related to different connection parameters.

4

Thanks Michael, I know that my next stop is Tower :slight_smile:
We are evaluating it.