# Downloading files with a specific regular expression from remote https host to target server local path. **URL:** \ **Category:** Ansible Project\ **Created:** [March 26, 2020, 6:24pm UTC](https://forum.ansible.com/t/downloading-files-with-a-specific-regular-expression-from-remote-https-host-to-target-server-local-path/32543 "2020-03-26T18:24:26Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![user106](https://avatars.discourse-cdn.com/v4/letter/u/41988e/32.png) [@user106](https://forum.ansible.com/u/user106)\ **Post date:** [March 26, 2020, 6:24pm UTC](https://forum.ansible.com/t/downloading-files-with-a-specific-regular-expression-from-remote-https-host-to-target-server-local-path/32543/1 "2020-03-26T18:24:26Z")
Having some difficulty finding what is the Ansible code to do this: Downloading files with a specific regular expression from remote http / https host to a target server local path. So would like to wget [http://remote.com/rpm/app\*.rpm](http://remote.com/rpm/app*.rpm) to a set of target servers. get\_url doesn’t appear to support patterns as per Ansible Documentation. On the other hand, with\_fileglob supports regular expressions and patterns but doesn’t work with HTTP. Looking for suggestions. Cheers, TK ---
**Author:** ![Dick\_Visser](https://avatars.discourse-cdn.com/v4/letter/d/d9b06d/32.png) [@Dick\_Visser](https://forum.ansible.com/u/Dick_Visser)\ **Post date:** [March 26, 2020, 11:14pm UTC](https://forum.ansible.com/t/downloading-files-with-a-specific-regular-expression-from-remote-https-host-to-target-server-local-path/32543/2 "2020-03-26T23:14:40Z")
It's fundamentally impossible to do what you want, unless the remote server offers some sort of file system equivalent, like a directory index. Dick ---
**Author:** ![user106](https://avatars.discourse-cdn.com/v4/letter/u/41988e/32.png) [@user106](https://forum.ansible.com/u/user106)\ **Post date:** [March 27, 2020, 1:09pm UTC](https://forum.ansible.com/t/downloading-files-with-a-specific-regular-expression-from-remote-https-host-to-target-server-local-path/32543/3 "2020-03-27T13:09:40Z")
Thanks Dick! I’ve started to get that impression after searching for quite some time. Currently using a shell command like this to get only specific files down: wget -r -nd --no-parent -A ‘_pattern_’ [http://site.com/path/to/file/](http://site.com/path/to/file/) Hence why I was thinking it might be possible in Ansible. Cheers, TK ---
**Author:** ![Dick\_Visser](https://avatars.discourse-cdn.com/v4/letter/d/d9b06d/32.png) [@Dick\_Visser](https://forum.ansible.com/u/Dick_Visser)\ **Post date:** [March 27, 2020, 5:51pm UTC](https://forum.ansible.com/t/downloading-files-with-a-specific-regular-expression-from-remote-https-host-to-target-server-local-path/32543/4 "2020-03-27T17:51:59Z")
The recursive (-r) option of wget only downloads files that are ‘visible’. This works fine for stuff like a web page with indexed directory listings etc. But anything that is not listed won’t be magically retrieved. If a site does not contain any links to content that is actually there, wget will not know about it and hence won’t download it. If wget works, then [http://remote.com/rpm](http://remote.com/rpm) must have links to all the files. So you best bet is to use the ansible command module with said wget options - provided you want to use ansible. Having said that, maybe you can elaborate on what the underlying task at hand is, and/or share the real/actual URLs etc. It might be possible to achieve the same thing in a different way. Dick ---
**Author:** ![user106](https://avatars.discourse-cdn.com/v4/letter/u/41988e/32.png) [@user106](https://forum.ansible.com/u/user106)\ **Post date:** [March 27, 2020, 7:25pm UTC](https://forum.ansible.com/t/downloading-files-with-a-specific-regular-expression-from-remote-https-host-to-target-server-local-path/32543/5 "2020-03-27T19:25:57Z")
Let’s assume a real site: [http://mirror.centos.org/centos/7/os/x86\_64/Packages/](http://mirror.centos.org/centos/7/os/x86_64/Packages/) `wget -r -nd --no-parent -A '*glib*' [http://mirror.centos.org/centos/7/os/x86_64/Packages/](http://mirror.centos.org/centos/7/os/x86_64/Packages/)` [Grabs all the packages that have glib in the name. Now you’re also right. If it’s not listed it won’t be downloaded. I’m talking about the listed packages.](http://mirror.centos.org/centos/7/os/x86_64/Packages/) So I used the shell command to run the wget, but I want to see if there is a pure ansible way of doing this. Example: ` # wget -r -nd --no-parent -A ‘_glib_’ [http://mirror.centos.org/centos/7/os/x86\_64/Packages/](http://mirror.centos.org/centos/7/os/x86_64/Packages/) # ls -altri|grep -v glib total 72652 201326721 dr-xr-x—. 9 root root 4096 Mar 27 15:04 .. 135239432 drwxr-xr-x. 2 root root 8192 Mar 27 15:07 . ---
**Author:** ![oxido\_A](https://avatars.discourse-cdn.com/v4/letter/o/7cd45c/32.png) [@oxido\_A](https://forum.ansible.com/u/oxido_A)\ **Post date:** [March 27, 2020, 7:51pm UTC](https://forum.ansible.com/t/downloading-files-with-a-specific-regular-expression-from-remote-https-host-to-target-server-local-path/32543/6 "2020-03-27T19:51:20Z")
well may you will need to read the output of curl and then grep over some html tags… I do some like this : curl grep -o -E ‘href=“([^”\*]+)"’ | cut -d ‘"’ -f 2 | sort -n | sed -e ‘s////’ | tail -1 this wil give you el name of the last file in the list of href=“([^”\*]+ in my case there are links… ---
**Author:** ![Dick\_Visser](https://avatars.discourse-cdn.com/v4/letter/d/d9b06d/32.png) [@Dick\_Visser](https://forum.ansible.com/u/Dick_Visser)\ **Post date:** [March 27, 2020, 8:15pm UTC](https://forum.ansible.com/t/downloading-files-with-a-specific-regular-expression-from-remote-https-host-to-target-server-local-path/32543/7 "2020-03-27T20:15:44Z")
if those files are tailor made for you then I would work with the team providing them to find an alternative way of doing this. What if they created a json file containing all the file information and key that with the host info, after the files are generated?