Lineinfile: loop generates multiple result lines

When running the below playbook for the first tile, all is OK. But as soon as you re-run it after fome changes in the playbook, it will generage additional result lines on changed lines.

- name: Editing text multiple lines with loop and variables from list
  lineinfile:
    path: /usr/local/bin/xxx/test
    regexp: "{{ item.regexp }}"
    line: "{{ item.line }}"
  loop:    #with_item: or loop:
    - { regexp: '^host name:', line: 'host name: {{ host_name }}' }
    - { regexp: '^description:', line: 'description: {{ host_description }}' }
    - { regexp: '^#?cpus:', line: 'cpus: {{ cpu_cores }}' }

First run result:

host name: ubuntu01
description: test
cpus: 24

Second run result after play book changes:

host name: ubuntu01
description: test
cpus: 24
cpus: 12

System adds additional “cpus:” lines after each change instead of changing the existing line. How to fix this issue and replace only without adding lines in text?

I tried your code and it works fine for me
it is updating the last value of cpu if changed…
im using

ansible==10.2.0
ansible-core==2.17.2

If I simply rerung without changes, all OK. As soon as I change something, it ads additional lines.

ansible [core 2.16.9]
config file = /home/johann75/ansible/ansible.cfg
configured module search path = [‘/home/johann75/.ansible/plugins/modules’, ‘/usr/share/ansible/plugins/modules’]
ansible python module location = /usr/lib/python3/dist-packages/ansible
ansible collection location = /home/johann75/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/bin/ansible
python version = 3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0] (/usr/bin/python3)
jinja version = 3.0.3
libyaml = True

I too failed to reproduce the behavior you report.

Are you certain you didn’t tweak anything in the process of posting the original task?

The task I used is a straight copy-n-paste from yours above with these changes:

  • I changed the path to /tmp/test
  • I added create: true (because I had no /tmp/test file initially)
  • I set these playbook variables thus:
  vars:
    host_name: tango
    host_description: I am a test host.
    cpu_cores: '{{ 12 | random | int }}' # to get different values on subsequent runs
utoddl@tango:~/ansible$ ansible --version
ansible [core 2.16.9]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/utoddl/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.12/site-packages/ansible
  ansible collection location = /home/utoddl/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.12.4 (main, Jun  7 2024, 00:00:00) [GCC 14.1.1 20240607 (Red Hat 14.1.1-5)] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True

Yes Tod, your code would work with a randome number with playbook unchanged. With a changing variable and unchanged playbook, no issues, but as soon as I change a looped line in the playbook, I will get a duplicate line on that changed on next run. And if I change againe, I would get the line tripple and so on. If I simply change a regular lineinfile: without a loop and modify any line, I do not have the issues with duplicate lines.

  lineinfile:
    path: /usr/local/bin/xxx/test
    regexp: "^cpus:"
    line: "cpus: {{ cpu_cores }}"

I’m sorry, I don’t understand what you mean.

!!!

You’ve inserted a caret at the beginning of your “line:”. “line:” isn’t a regex; it’s just text.
That’s why you’re getting lines added: You’re checking for lines starting with “cpus:”, but adding lines starting with “^cpus:”.

No issues when no loops are used. Whatever whenever is changed.

      lineinfile:
        path: /usr/local/bin/xxx/edited_file
        regexp: '^text 2:'
        line: 'text 2: changed text first or second run with changes or without'

edited_file result regardles of the changes in the play book, one line will remain, just the text will change:

text 2: changed text first or second run with changes or without

Now addid a loop: or with_item: into the play book:

lineinfile:
        path: /usr/local/bin/xxx/edited_file
        regexp: "{{ item.regexp }}"
        line: "{{ item.line }}"
      loop:
        - { regexp: '^text 2:', line: 'text 2: unchanged text first run' }

First run of the playbook results:

text 2: unchanged text first run

Making changes in the playbook after fist run:

lineinfile:
        path: /usr/local/bin/xxx/edited_file
        regexp: "{{ item.regexp }}"
        line: "{{ item.line }}"
      loop:
        - { regexp: '^text 2:', line: 'text 2: changed text second run' }

Second run result dublicates the line instead of replacing it:

text 2: unchanged text first run
text 2: changed text second run

All following changes in the playbook will add additional lines.

I’m still trying (and failing) to replicate the behavior you describe.

Since there’s no way a file can know whether you’ve changed your playbook, there must be some way to demonstrate this phenomenon in a single play. To that end, try this playbook (below) which I’ve called johann_s.yml. For me, regardless of whether I run all the tasks in one go like this:

$ ansible-playbook johann_s.yml -v

or in stages like this:

$ ansible-playbook johann_s.yml -v --tags=aaa
$ ansible-playbook johann_s.yml -v --tags=bbb
$ ansible-playbook johann_s.yml -v --tags=ccc
$ ansible-playbook johann_s.yml -v --tags=ddd

the output from the last task always contains the expected single line of text.

If the behavior you’re seeing is somehow specific to your environment, version, ansible.cfg, etc., then you might get different results. Otherwise, if you see a single line in the file after the last task, then try tweaking this playbook to demonstrate this phenomenon in a reproducible way that doesn’t require making changes between runs.

Here’s the johann_s.yml playbook:

---
# johann_s.yml
- name: Line in file tests
  hosts: localhost
  gather_facts: false
  vars:
    my_path: /tmp/foo-{{ ansible_version.string | d('bar') }}.txt
  tasks:

    # aaa
    - name: Tag aaa - truncate {{ my_path }} # noqa no-changed-when
      ansible.builtin.command: truncate --size=0 {{ my_path }}
      tags: aaa

    - name: Tag aaa - First run lineinfile
      ansible.builtin.lineinfile:
        path: "{{ my_path }}"
        regexp: '^text 2:'
        line: 'text 2: same text for tags "aaa" and "bbb"; tags: {{ ansible_run_tags | join(",") }}'
      tags: aaa

    - name: Tag aaa - First run cat # noqa no-changed-when
      ansible.builtin.command: cat {{ my_path }}
      tags: aaa

    # bbb
    - name: Tag bbb - Second run lineinfile
      ansible.builtin.lineinfile:
        path: "{{ my_path }}"
        regexp: '^text 2:'
        line: 'text 2: same text for tags "aaa" and "bbb"; tags: {{ ansible_run_tags | join(",") }}'
      tags: bbb

    - name: Tag bbb - Second run cat # noqa no-changed-when
      ansible.builtin.command: cat {{ my_path }}
      tags: bbb

    # ccc - Now add a "loop:" or "with_item:" into the play book:
    - name: Tag ccc - Third run lineinfile with loop
      ansible.builtin.lineinfile:
        path: "{{ my_path }}"
        regexp: "{{ item.regexp }}"
        line: "{{ item.line }}"
      loop:
        - { regexp: '^text 2:', line: 'text 2: from "ccc"; tags: {{ ansible_run_tags | join(",") }}' }
      tags: ccc

    - name: Tag ccc - Third run cat # noqa no-changed-when
      ansible.builtin.command: cat {{ my_path }}
      tags: ccc

    # ddd - Making changes in the playbook after first run:
    - name: Tag ddd - Fourth run lineinfile with loop
      ansible.builtin.lineinfile:
        path: "{{ my_path }}"
        regexp: "{{ item.regexp }}"
        line: "{{ item.line }}"
      loop:
        - { regexp: '^text 2:', line: 'text 2: from "ddd"; tags: {{ ansible_run_tags | join(",") }}' }
      tags: ddd

    - name: Tag ddd - Fourth run cat # noqa no-changed-when
      ansible.builtin.command: cat {{ my_path }}
      tags: ddd

You do not make any changes to your playbook, therefore t works.

I’m not sure of your use case, but why are you looping with lineinfile and not using a template? Aside from that, and maybe I’m missing context, but why not just match to the end of the line?

- name: Editing text multiple lines with loop and variables from list
  lineinfile:
    path: /usr/local/bin/xxx/test
    regexp: "{{ item.regexp }}"
    line: "{{ item.line }}"
  loop:    #with_item: or loop:
    - { regexp: '^host name:', line: 'host name: {{ host_name }}' }
    - { regexp: '^description:', line: 'description: {{ host_description }}' }
    - { regexp: '^cpus:(.*)$', line: 'cpus: {{ cpu_cores }}' }
 

I import lists from tables to inventory. To create templates, I would need to use loops. If I add (.*)$, it duplicates all lines and not only the line with changes…

No. Changing the playbook is not what’s causing the issue you’re seeing. The file does not know which playbook created it or modified it last. Believe me: I modified that playbook dozens of times with multiple runs before posting it, and it has yet to produce multiple lines for me.

In the complete example playbook I posted, the use of tags while making 4 separate ansible-playbook runs should reproduce the effect for you, at least as far as you’ve described it. Please look at the playbook I posted and try running it both ways I outlined. One, or both, or neither of those methods will show the undesired extra lines. In any case your result would be “interesting” to those of us trying to help you.

But just to be clear: if you can please post two complete playbooks, the first of which creates (if necessary) and populates the file, and the second of which is a modified version of the first which causes the undesired extra lines, then perhaps we’ll be able to duplicate the issue. So far, whatever is causing the problem for you has not been evident to the rest of us. So seeing two complete, minimal playbooks that demonstrate the problem when run one after the other would be a great step forward to understand what’s happening.