# How to handle lists of files and avoid unnecessary changes **URL:** \ **Category:** Ansible Project\ **Created:** [November 8, 2017, 7:47pm UTC](https://forum.ansible.com/t/how-to-handle-lists-of-files-and-avoid-unnecessary-changes/25964 "2017-11-08T19:47:08Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Marc\_Haber](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@Marc\_Haber](https://forum.ansible.com/u/Marc_Haber)\ **Post date:** [November 8, 2017, 7:47pm UTC](https://forum.ansible.com/t/how-to-handle-lists-of-files-and-avoid-unnecessary-changes/25964/1 "2017-11-08T19:47:08Z")
Hi, on Debian systems, one can configure repositories in an /etc/apt/sources.list.d directory with multiple files. I usually name the files after the distribution, so that, in a simplyfied example, I might have a file /etc/apt/sources.list.d/stable.list on systems running Debian stable, and /etc/apt/sources.list.d/testing.list on systems running Debian testing. In the real case, there are additional distribution-depending files (security, backports, local packages in different stages). A similiar setup can usually be found in /etc/yum.repos.d/ on Red Hat based systems. A local admin might choose to place additional files into those directories manually, for example adding special repositories for third-party software that doesn't come with the distribution. But back to the Debian case. When I change a system from testing to stable, I want the all testing\*.list to vanish, and the stable\*.list files to appear, unless they're already there, in which case I want them untouched. Locally placed files should also be untouched. My first approach was to concentrate all the ansible-managed \*.list files under a common prefix, zda, giving, for example, zda-unstable.list. I then wrote code to first remove all zda\*.list files (and their laegacy-named instances), and code to deploy the correct zda-foo.list file: - name: search for sources.list files   find:     paths: "/etc/apt/sources.list.d"     patterns: "zda-\*.list,exp-mc.list,sid-mc.list,sid-zg-stable-mc.list,sid-zg-unstable-mc.list,stretch-mc.list,stretch-security.list,stretch-zg-stable-mc.list,stretch-zg-unstable-mc.list,buster-mc.list,stretch-zg-stable-mc.list,stretch-security.list"   register: sourceslistfiles - name: delete sources.list files   file:     path: "{{ item.path }}"     state: absent   with\_items: "{{ sourceslistfiles.files }}" and finally code to roll out new list files: - name: include repositories   tags:           repos   include\_tasks:           "{{distribution}}/{{distribution\_version}}/repos.yml" $ cat roles/common/tasks/debian/stretch/repos.yml ---
**Author:** ![Kai\_Stian\_Olstad](https://avatars.discourse-cdn.com/v4/letter/k/e19adc/32.png) [@Kai\_Stian\_Olstad](https://forum.ansible.com/u/Kai_Stian_Olstad)\ **Post date:** [November 8, 2017, 8:58pm UTC](https://forum.ansible.com/t/how-to-handle-lists-of-files-and-avoid-unnecessary-changes/25964/2 "2017-11-08T20:58:32Z")
> When I change a system from testing to stable, I want the all > testing\*.list to vanish, and the stable\*.list files to appear, unless > they're already there, in which case I want them untouched. Locally > placed files should also be untouched. How you can implement this depends on the feature you need 1. one server having a several repo stable, testing and unstable 2. one server only having stable, testing or unstable. > My first approach was to concentrate all the ansible-managed \*.list > files under a common prefix, zda, giving, for example, > zda-unstable.list. > > I then wrote code to first remove all zda\*.list files (and their laegacy-named > instances), and code to deploy the correct zda-foo.list file: > > - name: search for sources.list files >   find: >     paths: "/etc/apt/sources.list.d" >     patterns: > "zda-\*.list,exp-mc.list,sid-mc.list,sid-zg-stable-mc.list,sid-zg-unstable-mc.list,stretch-mc.list,stretch-security.list,stretch-zg-stable-mc.list,stretch-zg-unstable-mc.list,buster-mc.list,stretch-zg-stable-mc.list,stretch-security.list" >   register: sourceslistfiles > - name: delete sources.list files >   file: >     path: "{{ item.path }}" >     state: absent >   with\_items: "{{ sourceslistfiles.files }}" To clean up legacy I would only run this once and after that make Ansible do the work. > and finally code to roll out new list files: > > - name: include repositories >   tags: >           repos >   include\_tasks: >           "{{distribution}}/{{distribution\_version}}/repos.yml" > > $ cat roles/common/tasks/debian/stretch/repos.yml > --- > - name: zda-stretch-mc.list >   tags: >   - repos >   - stretch >   copy: >     dest: /etc/apt/sources.list.d/zda-stretch-mc.list >     owner: root >     group: root >     mode: 0644 >     content: | >             deb [http://debian.debian.zugschlus.de/debian/](http://debian.debian.zugschlus.de/debian/) stretch main contrib >   notify: apt update > - name: zda-stretch-security.list >   tags: >   - repos >   - stretch >   copy: >     dest: /etc/apt/sources.list.d/zda-stretch-security.list >     owner: root >     group: root >     mode: 0644 >     content: | >             deb > [http://debian-security.debian.zugschlus.de/debian-security/](http://debian-security.debian.zugschlus.de/debian-security/) > stretch/updates main contrib >             deb [http://security.debian.org/](http://security.debian.org/) stretch/updates main contrib >   notify: apt update Feature 1 only need one variable, call it "repo\_type", this could contain stable, testing or unstable. Then your tasks would look something like this - apt\_repository:      repo: deb [http://debian.debian.zugschlus.de/debian/](http://debian.debian.zugschlus.de/debian/) stretch main contrib      filename: zda-stretch-mc      state: "{{ (repo\_type == 'stable') | ternary('present', 'absent') }}" State present creates the file, and absent remove the file so Ansible will clean up the files so nothing is left behind when you change repo\_type. You would need one for each repo and repo\_type. But with with\_items and facts you could probably make a loop to deal with most of them. With feature 2 you could have something like stable\_state, testing\_state and unstable\_state and set them to absent or present, and then just use that variable in state. Hopefully this make some sense. ---
**Author:** ![Marc\_Haber](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@Marc\_Haber](https://forum.ansible.com/u/Marc_Haber)\ **Post date:** [November 9, 2017, 10:17am UTC](https://forum.ansible.com/t/how-to-handle-lists-of-files-and-avoid-unnecessary-changes/25964/3 "2017-11-09T10:17:40Z")
> \> When I change a system from testing to stable, I want the all > \> testing\*.list to vanish, and the stable\*.list files to appear, unless > \> they're already there, in which case I want them untouched. Locally > \> placed files should also be untouched. > > How you can implement this depends on the feature you need > 1. one server having a several repo stable, testing and unstable > 2. one server only having stable, testing or unstable. 2, but with execptions. The general function that I want is "zap everything matching a glob, execpt files that would be written to by another task" > \> My first approach was to concentrate all the ansible-managed \*.list > \> files under a common prefix, zda, giving, for example, > \> zda-unstable.list. > \> > \> I then wrote code to first remove all zda\*.list files (and their > \> laegacy-named > \> instances), and code to deploy the correct zda-foo.list file: > \> > \> - name: search for sources.list files > \> find: > \> paths: "/etc/apt/sources.list.d" > \> patterns: > \> "zda-\*.list,exp-mc.list,sid-mc.list,sid-zg-stable-mc.list,sid-zg-unstable-mc.list,stretch-mc.list,stretch-security.list,stretch-zg-stable-mc.list,stretch-zg-unstable-mc.list,buster-mc.list,stretch-zg-stable-mc.list,stretch-security.list" > \> register: sourceslistfiles > \> - name: delete sources.list files > \> file: > \> path: "{{ item.path }}" > \> state: absent > \> with\_items: "{{ sourceslistfiles.files }}" > > To clean up legacy I would only run this once and after that make Ansible do > the work. I also want to clean up uncontrolled growth as a continuous process, and I want to be able to switch a host from one distribution to another with the files belonging to the "old" one cleaned up automatically. > Feature 1 only need one variable, call it "repo\_type", this could contain > stable, testing or unstable. > > Then your tasks would look something like this > > - apt\_repository: >     repo: deb [http://debian.debian.zugschlus.de/debian/](http://debian.debian.zugschlus.de/debian/) stretch main contrib >     filename: zda-stretch-mc >     state: "{{ (repo\_type == 'stable') | ternary('present', 'absent') }}" Probably a workaround if the really nice solution can not be had. The few hosts that have multple repositories would get the extra repos outside of ansible. I think I can live with that, but it's not pretty. > With feature 2 you could have something like stable\_state, testing\_state and > unstable\_state and set them to absent or present, and then just use that > variable in state. Also, rather ugly. But I have to admit that my sense of prettiness is not yet tuned to ansible. Greetings Marc ---
**Author:** ![Marc\_Haber](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@Marc\_Haber](https://forum.ansible.com/u/Marc_Haber)\ **Post date:** [November 9, 2017, 3:17pm UTC](https://forum.ansible.com/t/how-to-handle-lists-of-files-and-avoid-unnecessary-changes/25964/4 "2017-11-09T15:17:10Z")
This is what I did and it works. Here my solution: - name: search for sources.list files   find:     paths: "/etc/apt/sources.list.d"     patterns: "zda-\*.list,exp-mc.list,sid-mc.list,sid-zg-stable-mc.list,sid-zg-u   register: presentsourceslistfiles - name: set up fact list for wanted sources.list.d entries   set\_fact:     wantedsourceslistfiles: - name: include repositories   tags:           repos   include\_tasks:           "{{distribution}}/{{distribution\_version}}/repos.yml" - name: delete sources.list files   file:     path: "{{ item.path }}"     state: absent   with\_items: "{{ presentsourceslistfiles.files }}"   when: item.path not in wantedsourceslistfiles debian/sid/repos.yml: - name: create list of sid wantedsourceslistfiles   set\_fact:     sidwantedsourceslistfiles:       - /etc/apt/sources.list.d/zda-sid-mc.list       - /etc/apt/sources.list.d/zda-sid-zg-stable-mc.list       - /etc/apt/sources.list.d/zda-sid-zg-unstable-mc.list - name: add sid files to wantedsourceslistfiles   set\_fact:     wantedsourceslistfiles: "{{ wantedsourceslistfiles }} + {{ sidwantedsourceslistfiles }}" - name: zda-sid-mc.list   copy:     dest: /etc/apt/sources.list.d/zda-sid-mc.list     owner: root     group: root     mode: 0644     content: |             deb [http://debian.debian.zugschlus.de/debian/](http://debian.debian.zugschlus.de/debian/) sid main contrib   notify: apt update - name: zda-sid-zg-stable-mc.list   copy:     dest: /etc/apt/sources.list.d/zda-sid-zg-stable-mc.list     owner: root     group: root     mode: 0644     content: |             deb [http://zg20150.debian.zugschlus.de/zg20150/](http://zg20150.debian.zugschlus.de/zg20150/) sid-zg-stable main contrib   notify: apt update - name: zda-sid-zg-unstable-mc.list   copy:     dest: /etc/apt/sources.list.d/zda-sid-zg-unstable-mc.list     owner: root     group: root     mode: 0644     content: |             deb [http://zg20150.debian.zugschlus.de/zg20150/](http://zg20150.debian.zugschlus.de/zg20150/) sid-zg-unstable main contrib   notify: apt update Greetings Marc