# distributing files via several repos to balance load

**URL:** <https://forum.ansible.com/t/distributing-files-via-several-repos-to-balance-load/13514>\
**Category:** Ansible Project\
**Created:** [September 20, 2013, 1:54pm UTC](https://forum.ansible.com/t/distributing-files-via-several-repos-to-balance-load/13514 "2013-09-20T13:54:32Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Walid](https://avatars.discourse-cdn.com/v4/letter/w/ea5d25/32.png) [@Walid](https://forum.ansible.com/u/Walid)\
**Post date:** [September 20, 2013, 1:54pm UTC](https://forum.ansible.com/t/distributing-files-via-several-repos-to-balance-load/13514/1 "2013-09-20T13:54:32Z")

</div>

Dear All,

I was going through Will Thames Ansible examples to Brisbane devops day and I notice the configuration file copy, or file transfer pattern below:

```auto

- name: download java
  action: get_url url={{repo_url}}/{{java_archive}} dest={{tmpdir}}/{{java_archive}}

```

where repo\_url is defined as a group variable

```auto
*repo_url*: '[http://repo.dev.example:8000](http://repo.dev.example:8000)'

```

My Question : how can the repo\_url point to several URLS instead of one so that if there are several nodes executing it, and Ansible load balance the requests and know which URL to send back to which client ?

kind regards

Walid

---

<div class="post-metadata">

**Author:** ![James\_Cammarata1](https://avatars.discourse-cdn.com/v4/letter/j/d26b3c/32.png) [@James\_Cammarata1](https://forum.ansible.com/u/James_Cammarata1)\
**Post date:** [September 20, 2013, 2:24pm UTC](https://forum.ansible.com/t/distributing-files-via-several-repos-to-balance-load/13514/2 "2013-09-20T14:24:28Z")

</div>

You could set the URL in a host or group variable, so that it would be more evenly distributed. Beyond that, you could setup a reverse proxy like squid for your systems to fetch through so as soon as one system fetched the file you’d have a locally cached copy.

---

<div class="post-metadata">

**Author:** ![Walid](https://avatars.discourse-cdn.com/v4/letter/w/ea5d25/32.png) [@Walid](https://forum.ansible.com/u/Walid)\
**Post date:** [September 20, 2013, 2:42pm UTC](https://forum.ansible.com/t/distributing-files-via-several-repos-to-balance-load/13514/3 "2013-09-20T14:42:13Z")

</div>

Hi James,

the URL just as one example what if it was through a copy/fetch module not an http server but a file server, also could you elaborate on how one can extend the use of group variables to load balance source in automated way.

TIA

Walid

---

<div class="post-metadata">

**Author:** ![James\_Cammarata1](https://avatars.discourse-cdn.com/v4/letter/j/d26b3c/32.png) [@James\_Cammarata1](https://forum.ansible.com/u/James_Cammarata1)\
**Post date:** [September 20, 2013, 2:59pm UTC](https://forum.ansible.com/t/distributing-files-via-several-repos-to-balance-load/13514/4 "2013-09-20T14:59:54Z")

</div>

Yes, it would require splitting up your tasks and having your hosts in a second child group. For example, given an inventory file like this:

host1  
host2  
host3

[target1]  
host1

[target2]  
host2

[target3]  
host3

And group\_vars/target[1…3]:

target\_url: http://… # different url for each group var

You could have a playbook that does the download like this:

- get\_url: url={{target\_url}}/{{java\_archive}} dest={{tmpdir}}/{{java\_archive}}

---

<div class="post-metadata">

**Author:** ![Walid](https://avatars.discourse-cdn.com/v4/letter/w/ea5d25/32.png) [@Walid](https://forum.ansible.com/u/Walid)\
**Post date:** [September 20, 2013, 3:22pm UTC](https://forum.ansible.com/t/distributing-files-via-several-repos-to-balance-load/13514/5 "2013-09-20T15:22:23Z")

</div>

Thanks James, Will take this into consideration when scaling Ansible.

---

<div class="post-metadata">

**Author:** ![Brian\_Coca1](https://avatars.discourse-cdn.com/v4/letter/b/278dde/32.png) [@Brian\_Coca1](https://forum.ansible.com/u/Brian_Coca1)\
**Post date:** [September 20, 2013, 6:17pm UTC](https://forum.ansible.com/t/distributing-files-via-several-repos-to-balance-load/13514/6 "2013-09-20T18:17:43Z")

</div>

or you can just point to a load balancer and have it take care of it based on load

---

<div class="post-metadata">

**Author:** ![Walid](https://avatars.discourse-cdn.com/v4/letter/w/ea5d25/32.png) [@Walid](https://forum.ansible.com/u/Walid)\
**Post date:** [September 20, 2013, 8:04pm UTC](https://forum.ansible.com/t/distributing-files-via-several-repos-to-balance-load/13514/7 "2013-09-20T20:04:19Z")

</div>

Dear Brian,

I was looking more of a less intrusive internal Ansible solution without any considerations to the underlying physical infrastructure, however i do hear you and James, load balancer , caching, and probably DNS round robin are possible solutions. if it is not available internally by Ansible in a more automated way or data directive may be it will make it as a feature request later. Another configuration management solution does have it to answer problems of scale. the way it is done via 2-4 directives, something like select\_attribute directive that selects from a list and can apply automatically the division and allocation of source to destination uses an automatically created random weights based on a probability distribution if that makes any sense.

kind regards

Walid

---

<div class="post-metadata">

**Author:** ![Brian\_Coca1](https://avatars.discourse-cdn.com/v4/letter/b/278dde/32.png) [@Brian\_Coca1](https://forum.ansible.com/u/Brian_Coca1)\
**Post date:** [September 20, 2013, 8:34pm UTC](https://forum.ansible.com/t/distributing-files-via-several-repos-to-balance-load/13514/8 "2013-09-20T20:34:11Z")

</div>

One of the things I like about ansible is that it doesn't try to do  
everything itself, relies on existing well known and used solutions (ssh,  
sudo, cron, etc).

That is why some of us will push outside solutions (tcp load blancer, dns,  
etc) vs seeing it built into ansible. But it is perfectly understandable  
that other people have different preferences and would like to see more  
stuff 'built in'. It is very hard, if not impossible, to please everyone.

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\
**Post date:** [September 21, 2013, 1:20pm UTC](https://forum.ansible.com/t/distributing-files-via-several-repos-to-balance-load/13514/9 "2013-09-21T13:20:19Z")

</div>

Very much agree with Brian here, this is a _great_ case for using a load balancer.

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\
**Post date:** [September 21, 2013, 2:07pm UTC](https://forum.ansible.com/t/distributing-files-via-several-repos-to-balance-load/13514/10 "2013-09-21T14:07:15Z")

</div>

BTW, if not shared already, “with\_random\_choice” can be useful if you want something basic. I temporarily forgot about this 🙂

- debug: msg={{ item }}  
with\_random\_choice:
- boston
- paris
- tokyo

---

<div class="post-metadata">

**Author:** ![Walid](https://avatars.discourse-cdn.com/v4/letter/w/ea5d25/32.png) [@Walid](https://forum.ansible.com/u/Walid)\
**Post date:** [September 21, 2013, 3:32pm UTC](https://forum.ansible.com/t/distributing-files-via-several-repos-to-balance-load/13514/11 "2013-09-21T15:32:39Z")

</div>

Thanks Michael, and Brian, eventually we need to keep an open mind and adapt to tools and devop processes accordingly