How do I ensure firewall rule order

Get Help
1

ansible [core 2.19.2]
community.routeros 3.10.0

I want to ensure firewall rules match those in my vars/firewall.yml. I’m not sure how to specify this in the playbook.

What I have done is create a generic firewall.yml playbook task, and then I feed it with a loop.

It mostly works, without any attempt to pin the firewall rules in order. One rule always says it has changed, and results in a duplicate firewall entry added to the end of the firewall rules. The others are recognised as already being there, and are unchanged.

I tried using the following, but they don’t work with an Ansible loop. Only the last rule remains.

    handle_absent_entries: remove
    handle_entries_content: remove_as_much_as_possible
    ensure_order: true

tasks/firewall.yml

---
#/ip firewall filter
#  .id looks to be being ignored.
# Router is injecting a rule 0, so .id is always 1 off
# Rule with hw=yes always returned changed, and a duplicate rule is added.
- name: Firewall settings
  community.routeros.api_modify:
    path: ip firewall filter
    data:
      - action: "{{ item.action }}"
        chain: "{{ item.chain }}"
        comment: "{{ item.comment | default(omit) }}"
        address-list: "{{ item.address_list | default(omit) }}"
        address-list-timeout: "{{ item.address_list_timeout | default(omit) }}"
        connection-bytes: "{{ item.connection_bytes | default(omit) }}"
        connection-limit: "{{ item.connection_limit | default(omit) }}"
        connection-mark: "{{ item.connection_mark | default(omit) }}"
        connection-nat-state: "{{ item.connection_nat_state | default(omit) }}"
        connection-rate: "{{ item.connection_rate | default(omit) }}"
        connection-state: "{{ item.connection_state | default(omit) }}"
        connection-type: "{{ item.connection_type | default(omit) }}"
        content: "{{ item.content | default(omit) }}"
        disabled: "{{ item.disabled | default(omit) }}"
        dscp: "{{ item.dscp | default(omit) }}"
        dst-address: "{{ item.dst_address | default(omit) }}"
        dst-address-list: "{{ item.dst_address_list | default(omit) }}"
        dst-address-type: "{{ item.dst_address_type | default(omit) }}"
        dst-limit: "{{ item.dst_limit | default(omit) }}"
        dst-port: "{{ item.dst_port | default(omit) }}"
        fragment: "{{ item.fragment | default(omit) }}"
        hotspot: "{{ item.hotspot | default(omit) }}"
        hw-offload: "{{ item.hw_offload | default(omit) }}"
        icmp-options: "{{ item.icmp_options | default(omit) }}"
        in-bridge-port: "{{ item.in_bridge_port | default(omit) }}"
        in-bridge-port-list: "{{ item.in_bridge_port_list | default(omit) }}"
        in-interface: "{{ item.in_interface | default(omit) }}"
        in-interface-list: "{{ item.in_interface_list | default(omit) }}"
        ingress-priority: "{{ item.ingress_priority | default(omit) }}"
        ipsec-policy: "{{ item.ipsec_policy | default(omit) }}"
        ipv4-options: "{{ item.ipv4_options | default(omit) }}"
        jump-target: "{{ item.jump_target | default(omit) }}"
        layer7-protocol: "{{ item.layer7_protocol | default(omit) }}"
        limit: "{{ item.limit | default(omit) }}"
        log: "{{ item.log | default(omit) }}"
        log-prefix: "{{ item.log_prefix | default(omit) }}"
        nth: "{{ item.nth | default(omit) }}"
        out-bridge-port: "{{ item.out_bridge_port | default(omit) }}"
        out-bridge-port-list: "{{ item.out_bridge_port_list | default(omit) }}"
        out-interface: "{{ item.out_interface | default(omit) }}"
        out-interface-list: "{{ item.out_interface_list | default(omit) }}"
        p2p: "{{ item.p2p | default(omit) }}"
        packet-mark: "{{ item.packet_mark | default(omit) }}"
        packet-size: "{{ item.packet_size | default(omit) }}"
        per-connection-classifier: "{{ item.per_connection_classifier | default(omit) }}"
        port: "{{ item.port | default(omit) }}"
        priority: "{{ item.priority | default(omit) }}"
        protocol: "{{ item.protocol | default(omit) }}"
        psd: "{{ item.psd | default(omit) }}"
        random: "{{ item.random | default(omit) }}"
        realm: "{{ item.realm | default(omit) }}"
        reject-with: "{{ item.reject_with | default(omit) }}"
        routing-mark: "{{ item.routing_mark | default(omit) }}"
        routing-table: "{{ item.routing_table | default(omit) }}"
        src-address: "{{ item.src_address | default(omit) }}"
        src-address-list: "{{ item.src_address_list | default(omit) }}"
        src-address-type: "{{ item.src_address_type | default(omit) }}"
        src-mac-address: "{{ item.src_mac_address | default(omit) }}"
        src-port: "{{ item.src_port | default(omit) }}"
        tcp-flags: "{{ item.tcp_flags | default(omit) }}"
        tcp-mss: "{{ item.tcp_mss | default(omit) }}"
        time: "{{ item.time | default(omit) }}"
        tls-host: "{{ item.tls_host | default(omit) }}"
        ttl: "{{ item.ttl | default(omit) }}"
         # .id: "*{{ rindex + 1 }}"
  loop: "{{ firewall_rules }}"
  loop_control:
    index_var: rindex

vars/firewall.yml

# Note for variables with '-'s we have substituted '_' or we get an Ansible error with loops
firewall_rules:
  # input chain
  - { action: accept, chain: input, comment: "accept established,related,untracked", connection_state: "established,related,untracked" }
  - { action: drop, chain: input, comment: "drop invalid", connection_state: "invalid" }
  - { action: accept, chain: input, comment: "accept ICMP", protocol: icmp }
  - { action: accept, chain: input, comment: "accept to local loopback (for CAPsMAN)", dst_address: "127.0.0.1" }
  - { action: accept, chain: input, comment: "Allow Management Port", in_interface_list: "MGMT" }
  - { action: drop, chain: input, comment: "drop all not coming from LAN", in_interface_list: "!LAN" }
  # Forward chain
  - { action: accept, chain: forward, comment: "accept in ipsec policy", ipsec_policy: "in,ipsec" }
  - { action: accept, chain: forward, comment: "accept out ipsec policy", ipsec_policy: "out,ipsec" }
  # the following rule get duplicated every run
  - { action: "fasttrack-connection", chain: forward, comment: "fasttrack", connection_state: "established,related", hw_offload: "yes"}
  - { action: accept, chain: forward, comment: "accept established,related,untracked", connection_state: "established,related,untracked" }
  - { action: drop, chain: forward, comment: "drop invalid", connection_state: "invalid" }
  - { action: drop, chain: forward, comment: "drop all from WAN not DSTNATed", connection_nat_state: "!dstnat", connection_state: "new", in_interface_list: "WAN" }
nat_rules:
  - { action: masquerade, chain: srcnat, comment: "masquerade", ipsec_policy: "out,none", out_interface_list: "WAN" }

Gives:

TASK [networking : Firewall settings] *************************************************************************************************************************************************************************
ok: [fibre-02] => (item={'action': 'accept', 'chain': 'input', 'comment': 'accept established,related,untracked', 'connection_state': 'established,related,untracked'})
ok: [fibre-02] => (item={'action': 'drop', 'chain': 'input', 'comment': 'drop invalid', 'connection_state': 'invalid'})
ok: [fibre-02] => (item={'action': 'accept', 'chain': 'input', 'comment': 'accept ICMP', 'protocol': 'icmp'})
ok: [fibre-02] => (item={'action': 'accept', 'chain': 'input', 'comment': 'accept to local loopback (for CAPsMAN)', 'dst_address': '127.0.0.1'})
ok: [fibre-02] => (item={'action': 'accept', 'chain': 'input', 'comment': 'Allow Management Port', 'in_interface_list': 'MGMT'})
ok: [fibre-02] => (item={'action': 'drop', 'chain': 'input', 'comment': 'drop all not coming from LAN', 'in_interface_list': '!LAN'})
ok: [fibre-02] => (item={'action': 'accept', 'chain': 'forward', 'comment': 'accept in ipsec policy', 'ipsec_policy': 'in,ipsec'})
ok: [fibre-02] => (item={'action': 'accept', 'chain': 'forward', 'comment': 'accept out ipsec policy', 'ipsec_policy': 'out,ipsec'})
changed: [fibre-02] => (item={'action': 'fasttrack-connection', 'chain': 'forward', 'comment': 'fasttrack', 'connection_state': 'established,related', 'hw_offload': 'yes'})
ok: [fibre-02] => (item={'action': 'accept', 'chain': 'forward', 'comment': 'accept established,related,untracked', 'connection_state': 'established,related,untracked'})
ok: [fibre-02] => (item={'action': 'drop', 'chain': 'forward', 'comment': 'drop invalid', 'connection_state': 'invalid'})
ok: [fibre-02] => (item={'action': 'drop', 'chain': 'forward', 'comment': 'drop all from WAN not DSTNATed', 'connection_nat_state': '!dstnat', 'connection_state': 'new', 'in_interface_list': 'WAN'})

After 3 runs, the changedrule appears 3x
```
/ip firewall filter
add action=accept chain=input comment=“accept established,related,untracked” connection-state=established,related,untracked
add action=drop chain=input comment=“drop invalid” connection-state=invalid
add action=accept chain=input comment=“accept ICMP” protocol=icmp
add action=accept chain=input comment=“accept to local loopback (for CAPsMAN)” dst-address=127.0.0.1
add action=accept chain=input comment=“Allow Management Port” in-interface-list=MGMT
add action=drop chain=input comment=“drop all not coming from LAN” in-interface-list=!LAN
add action=accept chain=forward comment=“accept in ipsec policy” ipsec-policy=in,ipsec
add action=accept chain=forward comment=“accept out ipsec policy” ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment=fasttrack connection-state=established,related hw-offload=yes
add action=accept chain=forward comment=“accept established,related,untracked” connection-state=established,related,untracked
add action=drop chain=forward comment=“drop invalid” connection-state=invalid
add action=drop chain=forward comment=“drop all from WAN not DSTNATed” connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
add action=fasttrack-connection chain=forward comment=fasttrack connection-state=established,related hw-offload=yes
add action=fasttrack-connection chain=forward comment=fasttrack connection-state=established,related hw-offload=yes
add action=fasttrack-connection chain=forward comment=fasttrack connection-state=established,related hw-offload=yes
```

The complete playbook is at routeros_isp_modem. This works on a newly reset RB960 running routeros 7.19.4, with only the firewall rules needing to be corrected after a second run. A number of other tasks do report changed but these are not altering the configuration.

#firewall #get-help routeros #api_modify

2

This works if a list is passed to the data: field. It only fails if an Ansible loop is used.

- name: Firewall settings
community.routeros.api_modify:
handle_absent_entries: remove
handle_entries_content: remove_as_much_as_possible
ensure_order: true
path: ip firewall filter
data: “{{ firewall_rules2 }}

The router now only has the entries in firewall_rules2

Same works for the nat rules.

So as long as the firewall rules are listed in one variable, and nothing is dynamically added with other rules, this looks to be a solution to my problem.

3

Looks like loops don’t work well with most of the community.routeros.api calls, but you can use a loop before these calls to pre-build an Ansible fact, then use that fact in the routeros.api call. I haven’t had to do this for the firewall rules, but it has been helpful with other calls.

Setting facts with loops is more flexible too. I am building a config file for my routers, trying not to repeat the same entries (something common in a Mikrotik setup). I haven’t worked out the final form of the config file yet, but have gotten close.

e.g. using one or more loops to set the facts up, extracting the necessary fields from my yaml config file. This one uses a loop to build the interface bridge vlans. From the same yaml config, I also build the /interface vlan data field.

- name: build bridge vlans list
  ansible.builtin.set_fact:
    bridge_interface_vlans: >
      {{
        bridge_interface_vlans | default([]) +
        [
          item.1 | dict2items
          | selectattr('key', 'in', ['vlan-ids','tagged','untagged','comment'])
          | items2dict
          | combine( { 'bridge': item.0.name } )
        ]
      }}
  with_subelements:
    - "{{ bridge }}"
    - interface_vlans
    - skip_missing: True

- name: Configure VLANs on the bridge
  community.routeros.api_modify:
    handle_absent_entries: remove
    handle_entries_content: remove_as_much_as_possible
    path: "interface bridge vlan"
    data: "{{ bridge_interface_vlans }}"

- name: build vlan list
  ansible.builtin.set_fact:
    vlan_list: >
      {{
        vlan_list | default([])
        + [
            item.1
            | ansible.utils.remove_keys(['vlan-ids','ip_settings','dhcp_server','untagged','tagged'])
            | combine( { 'interface': item.0.name, 'vlan-id': item.1['vlan-ids'] } )
          ]
      }}
  with_subelements:
    - "{{ bridge }}"
    - interface_vlans
    - skip_missing: True

- name: Debug bridge list
  ansible.builtin.debug:
    var: vlan_list

- name: Add VLans to bridge
  community.routeros.api_modify:
    handle_absent_entries: remove
    handle_entries_content: remove_as_much_as_possible
    path: "interface vlan"
    data: "{{ vlan_list }}"

You can also add to a fact with multiple loops, so I build my ip settings up from loops over ethernet, bridge and vlan definitions in the yaml config.

These are all available in routeros ISP modem repo. Currently in the dev branch, but when I done, it will replace the current master branch.

4

I think this is a general problem with setting firewall rules (I remember similar challenges from using the community.general.ufw module), in particular when you have to use rather low-level tools such as community.routeros.api_modify (which is already a big step up from the other modules the collection provides).

If you want a specific order, and want other entries that are not controlled by you to be removed, you basically have to resort to compiling a list of all firewall rules that should be there, and use some module to batch-set them all (this is something community.routeros.api_modify can do). Which you already found out in the next post :slight_smile:

I’m not sure how this can be solved in a better way. If the firewall could be configured similarly to .d directories - you have one file per rule group, and the total config is assembled form alles files in that folder -, a better solution would exist. But that would be something you have to emulate somehow, since RouterOS doesn’t provide this (at least to my knowledge).

Also while finding a generic solution is quite hard, finding one that works well for some specific use-cases shouldn’t be too hard with some looping and combining some Jinja2 filters (and maybe writing own ones).

1 Like
5

The community.routeros.api project has done an amazing job at turning Mikrotik’s procedural configuration into an idempotent one. There are still some rough edges in my config, but it is way better than what I had before.