# Serving files from a central asset server **URL:** \ **Category:** Ansible Project\ **Created:** [February 28, 2014, 4:54am UTC](https://forum.ansible.com/t/serving-files-from-a-central-asset-server/14672 "2014-02-28T04:54:21Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Jason\_Sheedy](https://avatars.discourse-cdn.com/v4/letter/j/48db29/32.png) [@Jason\_Sheedy](https://forum.ansible.com/u/Jason_Sheedy)\ **Post date:** [February 28, 2014, 4:54am UTC](https://forum.ansible.com/t/serving-files-from-a-central-asset-server/14672/1 "2014-02-28T04:54:21Z")
Hi guys, I’m quite new to Ansible and would like to perform the following in an idempotent fashion. I would basically like to minimise the number of downloads I need to perform from the web for certain file assets, so my plan is to download them once to an asset server and then retrieve them from there for all the target hosts in our cluster. As far as I know the flow would go something like this. # does file exist on target host # if yes exit # if no retrieve from asset server # does file exist on asset server # if not download from web to asset server # copy file from asset server to target host Are there any modules I could use to do this or are there any other recommendations on best practices for this scenario? Any help would be appreciated. Cheers, Jason ---
**Author:** ![James\_Martin2](https://avatars.discourse-cdn.com/v4/letter/j/b9e5f3/32.png) [@James\_Martin2](https://forum.ansible.com/u/James_Martin2)\ **Post date:** [February 28, 2014, 3:12pm UTC](https://forum.ansible.com/t/serving-files-from-a-central-asset-server/14672/2 "2014-02-28T15:12:40Z")
Sounds like you'd have an asset server role that gets executed against the asset server that collects all the assets, followed by a role that gets executed against "target host". Take a look at the get\_url module for getting files from the web to your asset server, and get\_url or synchronize method to pull them from your asset server to your target host. - James ---
**Author:** ![Michael\_DeHaan1](https://avatars.discourse-cdn.com/v4/letter/m/5e9695/32.png) [@Michael\_DeHaan1](https://forum.ansible.com/u/Michael_DeHaan1)\ **Post date:** [February 28, 2014, 3:37pm UTC](https://forum.ansible.com/t/serving-files-from-a-central-asset-server/14672/3 "2014-02-28T15:37:10Z")
Yep, I’d definitely recommend a local play to update things, or fetching from Jenkins (etc) versus the internet. get\_url also takes a SHA as an option to avoid repeated downloads (though you might … just might … be interested to moving to packages instead!) ---
**Author:** ![Adam\_Morris](https://avatars.discourse-cdn.com/v4/letter/a/ecae2f/32.png) [@Adam\_Morris](https://forum.ansible.com/u/Adam_Morris)\ **Post date:** [February 28, 2014, 5:04pm UTC](https://forum.ansible.com/t/serving-files-from-a-central-asset-server/14672/4 "2014-02-28T17:04:52Z")
the copy module will also check if the file exists on the target server and not copy it if it is already there… CAVEAT, if you intend to modify the file on the target server then it will be replaced with the original if any of these modules detect that it is different. (i.e it will checksum the two files and only if they don’t match will it copy the source over. Adam Adam ---
**Author:** ![bryan\_hunt1](https://avatars.discourse-cdn.com/v4/letter/b/ed8c4c/32.png) [@bryan\_hunt1](https://forum.ansible.com/u/bryan_hunt1)\ **Post date:** [February 28, 2014, 10:34pm UTC](https://forum.ansible.com/t/serving-files-from-a-central-asset-server/14672/5 "2014-02-28T22:34:03Z")
I’m always in favor of using the right tool for the job, is this not the sort of task at which rsync excels? An rsync module perhaps? ---
**Author:** ![Brian\_Coca1](https://avatars.discourse-cdn.com/v4/letter/b/278dde/32.png) [@Brian\_Coca1](https://forum.ansible.com/u/Brian_Coca1)\ **Post date:** [February 28, 2014, 10:47pm UTC](https://forum.ansible.com/t/serving-files-from-a-central-asset-server/14672/6 "2014-02-28T22:47:12Z")
the syncronize module is an rsync wrapper. ---
**Author:** ![Jason\_Sheedy](https://avatars.discourse-cdn.com/v4/letter/j/48db29/32.png) [@Jason\_Sheedy](https://forum.ansible.com/u/Jason_Sheedy)\ **Post date:** [March 1, 2014, 10:19am UTC](https://forum.ansible.com/t/serving-files-from-a-central-asset-server/14672/7 "2014-03-01T10:19:45Z")
Thanks for the feedback guys. I think, as James suggested, the simplest way would be to download all the assets in a separate role. That sayid, I was hoping to download / sync to the asset server in an addhock fashion in various roles. To do that I think it would be easier to write a custom module than piecing together a long list of calls to standard modules every time I need an asset. As an example, I’m thinking something like this would be good: tasks - name: download asset and sync to asset server url: [http://www.some.com/asset.tar.gz](http://www.some.com/asset.tar.gz) dest: /var/assets asset\_host: root@my.asset.server This would download the asset if if doesn’t exist on the asset server and syncronise with the local file system. The dest directory could be mirrored between the asset server and the host machine. ---
**Author:** ![Dmitry\_Horbach](https://avatars.discourse-cdn.com/v4/letter/d/c6cbf5/32.png) [@Dmitry\_Horbach](https://forum.ansible.com/u/Dmitry_Horbach)\ **Post date:** [March 2, 2014, 10:00am UTC](https://forum.ansible.com/t/serving-files-from-a-central-asset-server/14672/8 "2014-03-02T10:00:49Z")
Custom module will work best in your case. We are using the same approach but with more sources of asset. This module is basically used in every software installation role and separate role is not an option. 1. Check if file is already present in destination directory 2. Download from /opt/repository/app (mount of software repository for installations with Vagrant) 3. Download from [http://reposerver/app](http://reposerver/app) or s3://repobucket/app 4. Download from internet - name: Get Packer installer get\_installer: \> name={{ packer\_installer }} dest={{ packer\_install\_home }} repo={{ repo\_app\_base }} path={{ packer\_repo\_path }} repo\_url={{ repo\_app\_url }} url={{ packer\_url }} sudo: yes Where packer\_installer: ‘{{ packer\_version }}\_linux\_386.zip’ packer\_repo\_path: HashiCorp/Packer/{{ packer\_version }} packer\_url: [https://dl.bintray.com/mitchellh/packer/](https://dl.bintray.com/mitchellh/packer/){{ packer\_installer }} repo\_app\_base: /opt/repository/app repo\_app\_url: [http://reposerver/app](http://reposerver/app) (or s3://repobucket/app) So module searches in 1. /opt/install/packer/0.5.1\_linux\_386.zip (destination) 2. /opt/repository/app/HashiCorp/Packer/0.5.1/0.5.1\_linux\_386.zip (nfs mount) 3. [http://reposerver/app/HashiCorp/Packer/0.5.1/0.5.1\_linux\_386.zip](http://reposerver/app/HashiCorp/Packer/0.5.1/0.5.1_linux_386.zip) (http server) or s3://repobucket/app/HashiCorp/Packer/0.5.1/0.5.1\_linux\_386.zip 4. [https://dl.bintray.com/mitchellh/packer/0.5.1\_linux\_386.zip](https://dl.bintray.com/mitchellh/packer/0.5.1_linux_386.zip) Drawback - none of your roles can be shared in Ansible Galaxy. ---
**Author:** ![Jason\_Sheedy](https://avatars.discourse-cdn.com/v4/letter/j/48db29/32.png) [@Jason\_Sheedy](https://forum.ansible.com/u/Jason_Sheedy)\ **Post date:** [March 2, 2014, 7:41pm UTC](https://forum.ansible.com/t/serving-files-from-a-central-asset-server/14672/9 "2014-03-02T19:41:57Z")
Thanks Dmitry. Hosting it on s3 is a great idea. We may be able to use that with get\_url as a short term solution. The download speed is the main issue and we get great speeds from s3. Cheers