Ansible playbook for updated file listing

Get Help
,
1

I am trying to develop an ansible module which would traverse subfolders under a main folder and list all .SQL files inside the sub-folders which were modified in the last 30 minutes. Below is my code:

  • name: Install SQL scripts on Private PostgreSQL Instance
    hosts: localhost
    gather_facts: no

become: yes

tasks:
- name: install pip3
apt: name=python3-pip state=present

- name: Install required apt packages
  apt:
    name: 
      - libpq-dev
      
- name: Install required python packages
  pip:
    name: 
      - psycopg2

- name: Download cloudSQL proxy
  ansible.builtin.get_url:
    url: https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.13.0/cloud-sql-proxy.linux.amd64
    dest: ./cloud-sql-proxy
    mode: "0755"

- name: Set GCP project
  command:
    gcloud config set project '{{ gcp_project }}'

- name: Print gcloud config
  command:
    gcloud config list
    
- name: Start cloudSQL proxy
  command: ./cloud-sql-proxy --private-ip "{{ gcp_project }}:{{ db_instance_location }}:{{ db_instance_name }}"
  async: 3600
  poll: 0


- name: List subfolders
  find: 
    path: DDL/
    recurse: no
    file_type: directory
  register: subfolders

- name: Find recently modified files in each subfolder
  find:
    path: "{{ item.path }}"
    recurse: yes
    patterns: "*.sql"
    age: "-30m"
    age_stamp: "mtime"
    use_regex: yes
  loop: "{{ subfolders.files }}"
  register: recent_files_per_subfolder
  loop_control:
    label: "{{ item.path }}"

- name: Display files modified within the last 5 minutes for each subfolder
  debug:
    msg: "Files modified in {{ item.item.path }}: {{ item.files | map(attribute='path') | list }}"
  loop: "{{ recent_files_per_subfolder.results }}"
  when: subfolders.files | length > 0

However the playbook output not listing anything. I have tried changing the age parameter, but the output is same.

Any help would be really appreciated.

2

I’m not completely sure, but patterns: "*.sql" and use_regex: yes doesn’t match, as *.sql is a filename-expansion pattern and not a regular expression.
For it to be a regexp it would need to be something like ^.*\.sql$

HTH

2 Likes
3

This solution doesn’t seem to work.

4

What does subfolders look like, try adding a debug for this and post the result here?

- name: Debug subfolders
  ansible.builtin.debug:
    var: subfolders

Also try setting use_regex to false as *.sql is a shell pattern?

5

Hi Chris,
Here is what the folder structure looks like:

DDL(main folder) → Multiple sub-folders → Each sub-folder contains multiple .sql files

6

Right, but does the subfolders dictionary contain the list of sub-directories? I’m asking as DDL/ is a relative path and I’m wondering if a absolute path might be needed here? :person_shrugging:

7

yes it does:

- name: List subfolders
  find: 
    path: DDL/
    recurse: yes
    file_type: directory
  register: subfolders

- name: Find recently modified files in each subfolder
  find:
    path: "{{ item.path }}"
    recurse: yes
    patterns: "*.sql"
    age: "-10m"

age_stamp: “mtime”

use_regex: yes

  loop: "{{ subfolders.files }}"
  register: recent_files_per_subfolder
  loop_control:
    label: "{{ item.path }}"

- name: Display files modified within the last 5 minutes for each subfolder
  debug:
    msg: "Files modified in {{ item.item.path }}: {{ item.files | map(attribute='path') | list }}"
  loop: "{{ recent_files_per_subfolder.results }}"
  when: subfolders.files | length > 0
8

Did you try use_regex: false as I suggested above?