# Replace unmanaged text block with managed block? **URL:** \ **Category:** Get Help\ **Created:** [October 4, 2024, 7:56pm UTC](https://forum.ansible.com/t/replace-unmanaged-text-block-with-managed-block/9691 "2024-10-04T19:56:05Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![HiPhish](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/hiphish/32/4901_2.png) [@HiPhish](https://forum.ansible.com/u/HiPhish)\ **Post date:** [October 4, 2024, 7:56pm UTC](https://forum.ansible.com/t/replace-unmanaged-text-block-with-managed-block/9691/1 "2024-10-04T19:56:05Z")
Hello everyone, I would like to replace an unmanaged block of text from a configuration file (for Dovecot) with a managed block. The difficulty is that the two blocks have different content, so I thought of splitting it up into two steps: first find and remove the old block, then add the new block using the `ansible.builtin.blockinfile` module. The second part is easy, but I cannot figure out how to remove a multi-line string. Here is what the original (unmanaged) block looks like: ```auto service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { #mode = 0666 } # Create inet listener only if you can't use the above UNIX socket #inet listener lmtp { # Avoid making LMTP visible for the entire internet #address = #port = #} } ``` It would be enough to match the first (the opening `^service lmtp {`) and last (the closing `}`) line and include everything in between. However, I have not been able to find out how to use `ansible.builtin.replace` to match this entire content. My idea was to use this: ```auto - name: 'Remove default LMTP configuration' ansible.builtin.replace: path: '/etc/dovecot/conf.d/10-master.conf' before: '# BEGIN ANSIBLE MANAGED LMTP CONFIGURATION' regexp: '^service lmtp \{\n(.*\n)*\n^\}' ``` Since `.` does not match newlines I use `^service lmtp \{\n` (explicit new line), followed by an arbitrary amount of of lines and ending with `^\}` (a closing curly brace), making use of the fact that the closing `}` is not indented. However, the regex does not match and the old block remains in the file. And even if it did I would have to make the match non-greedy, otherwise I would delete up to the last unindented `}` of the entire file. Is there a way to write a multiline regex for `replace`? Or maybe I am going at this completely the wrong way and there is a smarter solution? ---
**Author:** ![utoddl](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/utoddl/32/249_2.png) [@utoddl](https://forum.ansible.com/u/utoddl)\ **Post date:** [October 5, 2024, 1:05am UTC](https://forum.ansible.com/t/replace-unmanaged-text-block-with-managed-block/9691/2 "2024-10-05T01:05:52Z")
According to [https://docs.ansible.com/ansible/latest/collections/ansible/builtin/replace\_module.html#parameter-regexp](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/replace_module.html#parameter-regexp), > The regular expression to look for in the contents of the file. > Uses Python regular expressions; see [re — Regular expression operations — Python 3.12.7 documentation](https://docs.python.org/3/library/re.html). > Uses MULTILINE mode, which means `^` and `$` match the beginning and end of the file, as well as the beginning and end respectively of _each line_ of the file. > Does not use DOTALL, which means the `.` special character matches any character _except newlines_. A common mistake is to assume that a negated character set like `[^#]` will also not match newlines. > In order to exclude newlines, they must be added to the set like `[^#\n]`. Because `regex:` doesn’t use `DOTALL`, you have to use “extra stuff” to express a dot that includes newlines. It looks like you were trying to do that. I used the following expression on your data with success: ```yml regexp: '^service lmtp \{(.|\n)*?^}$' ``` Note the trailing `$`, which you missed. But the real reason your tasks would never match is because of your `before:` parameter. `before:` doesn’t specify text that `ansible.builtin.replace` will insert before the replacement. Rather, it restricts the scope of the replacement action to the part of the file which appears _before_ the specified string. If your file doesn’t already contain `# BEGIN ANSIBLE MANAGED LMTP CONFIGURATION`, then that task isn’t going to affect that file. Here’s a complete working example: ```yaml --- # HiPhish.yml - name: Replace example hosts: localhost gather_facts: false vars: filename: HiPhish.txt tasks: - name: Create temp file ansible.builtin.copy: dest: '{{ filename }}' content: | stuff before service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { #mode = 0666 } # Create inet listener only if you can't use the above UNIX socket #inet listener lmtp { # Avoid making LMTP visible for the entire internet #address = #port = #} } stuff after - name: Use Replace ansible.builtin.replace: path: '{{ filename }}' regexp: '^service lmtp \{(.|\n)*?^}$' replace: | ### Start of replaced text ### Ha ha! Gotchya. ### End of replaced text ### ``` ---
**Author:** ![HiPhish](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/hiphish/32/4901_2.png) [@HiPhish](https://forum.ansible.com/u/HiPhish)\ **Post date:** [October 5, 2024, 9:29am UTC](https://forum.ansible.com/t/replace-unmanaged-text-block-with-managed-block/9691/3 "2024-10-05T09:29:26Z")
Thank you for the regex, it works. > But the real reason your tasks would never match is because of your `before:` parameter. `before:` doesn’t specify text that `ansible.builtin.replace` will insert before the replacement. Rather, it restricts the scope of the replacement action to the part of the file which appears _before_ the specified string. If your file doesn’t already contain `# BEGIN ANSIBLE MANAGED LMTP CONFIGURATION`, then that task isn’t going to affect that file. The reason I was using `before` is for idempotency: after deleting the unmanaged block the next task will append the managed block at the end. On a subsequent run I do not want `replace` to delete the managed block, hence I was using `before` as a way to say “not after this string”. Is there a better way of telling Ansible to not delete after the marker line? ---
**Author:** ![utoddl](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/utoddl/32/249_2.png) [@utoddl](https://forum.ansible.com/u/utoddl)\ **Post date:** [October 5, 2024, 6:14pm UTC](https://forum.ansible.com/t/replace-unmanaged-text-block-with-managed-block/9691/4 "2024-10-05T18:14:38Z")
> [@HiPhish](#): > > The reason I was using `before` is for idempotency: after deleting the unmanaged block the next task will append the managed block at the end. On a subsequent run I do not want `replace` to delete the managed block, hence I was using `before` as a way to say “not after this string”. Okay, but the task in your [original post](http://forum.ansible.com/t/replace-unmanaged-text-block-with-managed-block/9691) in the topic, named “`Remove default LMTP configuration`” won’t do what it says because “`# BEGIN ANSIBLE MANAGED LMTP CONFIGURATION`” isn’t in `/etc/dovecot/conf.d/10-master.conf`. So you have a chicken-and-egg problem. If this were my problem, I’d break it into several steps. 1. Use [`ansible.builtin.lineinfile`](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/lineinfile_module.html#ansible-builtin-lineinfile-module-manage-lines-in-text-files) to loop over `'# BEGIN ANSIBLE MANAGED LMTP CONFIGURATION'` and `'# END ANSIBLE MANAGED LMTP CONFIGURATION'`, making sure that if they don’t already exist in the file that they get appended. 2. Use the task “`Remove default LMTP configuration`” from your original post. Since the “`# BEGIN […]`” line is now there, that task’s `before:` parameter should do the Right Thing™. 3. Use [`ansible.builtin.replace`](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/replace_module.html#ansible-builtin-replace-module-replace-all-instances-of-a-particular-string-in-a-file-using-a-back-referenced-regular-expression) to replace everything between the “`# BEGIN`” and “`# END`” lines. You can use the `before:` parameter to indicate the “`# END`” line, and the `after:` parameter to indicate the “`# BEGIN`”. The `regex:` parameter is required, so `(.|\n)*` should get everything between the two, including nothing. You may be tempted to omit the “`# END`” line since you won’t have anything after it, but that would be a mistake. If you find that later you need to replace existing or add not-yet-existing configuration and put it under Ansible management, having an explicit beginning and ending of each such section would be handy. This whole house of cards assumes the `dovecot` package (or whatever created `'/etc/dovecot/conf.d/10-master.conf'`) isn’t going to come along and replace it later with an upgrade or phase of the moon change or who knows what else. Some common config files (like `/etc/passwd`) are changed in well-understood ways that multiple changing entities are aware of so they generally don’t step on each other’s work. I wouldn’t comfortably make that assumption in the case of `conf.d/*.conf` files for random applications. You also asked if maybe there’s a better way to go about it, or a smarter solution. I don’t know about `dovecot` in particular, but in general, if an application supports the `…/app/conf.d/##-name.conf` paradigm, you’re better off leaving installed files alone and adding/overriding config with your own `conf.d/##-name.conf` files. If you define another “`service lmtp {…}`” in a conf file that’s processed after `'/etc/dovecot/conf.d/10-master.conf'`, it’s possible that later redefinition would override the original, in which case you need not mess with the `10-master.conf` file at all. Then you could build, say, `'/etc/dovecot/conf.d/99-local-ansible.conf'` through an Ansible template. That would be far preferable _if that works_. You would need to test of course to see. ---
**Author:** ![HiPhish](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/hiphish/32/4901_2.png) [@HiPhish](https://forum.ansible.com/u/HiPhish)\ **Post date:** [October 8, 2024, 6:53pm UTC](https://forum.ansible.com/t/replace-unmanaged-text-block-with-managed-block/9691/5 "2024-10-08T18:53:35Z")
Oh wow, I feel so stupid in hindsight. Yes, Dovecot does support exactly what you describe and it’s a much better solution than editing existing files. Not just easier to implement in Ansible, but also easier to maintain and undo if necessary.