issues with ansible looping

Archives Ansible Project
1

Hi Team,

I’m having issues with looping in Ansible. The background of what I’m trying to do is -
I have 2 regions in aws_cloud_regions and their respective waf_blocked_accounts list, which looks like the one below.

I want to create regional_account_rules in waf for both the regions (as in us-east-1 blocked_account_list gets attached to regional_account_rules for US East and the same for another region) but facing issues while looping over regions and blocked_account_list together.

Also note, that search_string in “Create statements” accepts a string list, so we have to create one outer loop and one inner loop, an outer loop for regions, and an inner for adding blocked account lists one by one.

-bash-4.2$ cat environment/QAtest/us-east-1/waf_blocked_accounts.yml

blocked_account_list:

  • 5afabfb36d6c356772d8ae02

  • 5c46e33273766a3634f91a7c

“aws_cloud_regions”: [
“us-east-1”,
“eu-central-1”
]

The playbook which needs modification, it is not region-specific as of now:

  • name: Loop over AWS regions

include_vars:

file: “environment/QAtest/{{ region }}/waf_blocked_accounts.yml”

loop: “{{ aws_cloud_regions }}”

loop_control:

loop_var: region

register: blocked_accounts

  • name: Create statements

set_fact:

acc_statements: “{{ acc_statements + [loop_statement] }}”

vars:

loop_statement:

byte_match_statement:

search_string: “{{ acc_id }}”

positional_constraint: EXACTLY

field_to_match:

single_header:

name: “accountmoid”

text_transformations:

  • type: NONE

priority: 0

loop: “{{ blocked_account_list }}”

loop_control:

loop_var: acc_id

  • set_fact:

regional_account_rules:

  • name: “BlockedAccounts”

priority: 3

action:

block: {}

visibility_config:

sampled_requests_enabled: yes

cloud_watch_metrics_enabled: yes

metric_name: “BlockedAccounts”

statement:

or_statement:

statements: “{{ acc_statements }}”

  • set_fact:

regional_account_rules: “{{ regional_account_rules | default() }}”

Any help is appreciated. Thanks in advance.

2

Your first task is replacing blocked_account_list each time through the loop, so you end up with only those blocked accounts listed for the last region.

However, you are also registering the results, so you can create a loop that retains all the blocked accounts along with their associated region.

    - name: Tie region to the blocked accounts
      ansible.builtin.debug:
        msg: "{{ item }}"
      vars:
        ba_query: '[].{region: region, blocked_accounts: ansible_facts.blocked_account_list}'
      loop:
        - "{{ blocked_accounts.results | json_query(ba_query) }}"

This result in the following output. (Note, I’m running with
ANSIBLE_STDOUT_CALLBACK=yaml ansible-playbook …
and I’ve inserted the region into the account numbers so I can tell which accounts came from which region.):

TASK [Tie region to the blocked accounts] ************************************************************************************************************************************
ok: [localhost] => (item=[{'region': 'us-east-1', 'blocked_accounts': ['20ea8d-us-east-1-bfbafa5', 'c7a19f-us-east-1-33e64c5']}, {'region': 'eu-central-1', 'blocked_accounts': ['5afabf-eu-central-1-ae02', '5c46e3-eu-central-1-1a7c']}]) => 
  msg:
  - blocked_accounts:
    - 20ea8d-us-east-1-bfbafa5
    - c7a19f-us-east-1-33e64c5
    region: us-east-1
  - blocked_accounts:
    - 5afabf-eu-central-1-ae02
    - 5c46e3-eu-central-1-1a7c
    region: eu-central-1

After that, it isn’t particularly clear to me how the region is supposed to play into the following tasks. But perhaps this will help get past the first problem.

3

Thanks, Todd. I’m majorly facing issues with looping. I want to create regional_account_rules for us-east-1 and eu-central-1 and want to make sure correct “acc_statements” get created for each region with respective blocked accounts. Could you provide some suggestions how to achieve this?

4

The code I have written is:

  • name: Populate acc_statements
    set_fact:
    acc_statements: “{{ acc_statements | combine({acc_id.region: acc_statements[acc_id.region] | default() + [loop_statement]}) }}”
    vars:
    loop_statement:
    byte_match_statement:
    search_string: “{{ acc_id.ansible_facts.blocked_account_list }}”
    positional_constraint: EXACTLY
    field_to_match:
    single_header:
    name: “accountmoid”
    text_transformations:

  • type: NONE
    priority: 0
    loop: “{{ blocked_accounts.results }}”
    loop_control:
    loop_var: acc_id

  • debug:
    var: acc_statements

The output I’m getting:

“acc_statements”: {
“eu-central-1”: [
{
“byte_match_statement”: {
“field_to_match”: {
“single_header”: {
“name”: “accountmoid”
}
},
“positional_constraint”: “EXACTLY”,
“search_string”: [
“5afabfb36d6c356772d84362”,
“5c46e33273766a3634f91a8d”
],
“text_transformations”: [
{
“priority”: 0,
“type”: “NONE”
}
]
}
}
],
“us-east-1”: [
{
“byte_match_statement”: {
“field_to_match”: {
“single_header”: {
“name”: “accountmoid”
}
},
“positional_constraint”: “EXACTLY”,
“search_string”: [
“5afabfb36d6c356772d8ae02”,
“5c46e33273766a3634f91a7c”
],
“text_transformations”: [
{
“priority”: 0,
“type”: “NONE”
}
]
}
}
]
}
}

And the output I want is, search_string should be a string instead of a list:

“acc_statements”: {
“eu-central-1”: [
{
“byte_match_statement”: {
“field_to_match”: {
“single_header”: {
“name”: “accountmoid”
}
},
“positional_constraint”: “EXACTLY”,
“search_string”: “5afabfb36d6c356772d84362”,
“text_transformations”: [
{
“priority”: 0,
“type”: “NONE”
}
]
}
},
{
“byte_match_statement”: {
“field_to_match”: {
“single_header”: {
“name”: “accountmoid”
}
},
“positional_constraint”: “EXACTLY”,
“search_string”: “5c46e33273766a3634f91a8d”,
“text_transformations”: [
{
“priority”: 0,
“type”: “NONE”
}
]
}
}
],
“us-east-1”: [
{
“byte_match_statement”: {
“field_to_match”: {
“single_header”: {
“name”: “accountmoid”
}
},
“positional_constraint”: “EXACTLY”,
“search_string”: “5afabfb36d6c356772d8ae02”
“text_transformations”: [
{
“priority”: 0,
“type”: “NONE”
}
]
}
}
]
}
}

5

You are close, and your intuition is correct that there’s a way to loop through with one item per (region × account). This is exactly the situation that the subelements filter is designed for. Try this:

    - name: Populate acc_statements
      ansible.builtin.set_fact:
        acc_statements: "{{ acc_statements | default([]) | combine({acc_id**[0]**.region: acc_statements[acc_id**[0]**.region] | default([]) + [loop_statement]}) }}"
      vars:
        loop_statement:
          byte_match_statement:
            search_string: "{{ acc_id**[1]** }}"
            positional_constraint: EXACTLY
            field_to_match:
              single_header:
                name: "accountmoid"
            text_transformations:
              - type: NONE
                priority: 0
      loop: "{{ blocked_accounts.results **| subelements('ansible_facts.blocked_account_list')** }}"
      loop_control:
        loop_var: acc_id

The subelements filter takes a list (blocked_accounts.results in this case) and duplicates each item once per each subelement of the item as referenced by the given string. Each element of the list it returns (acc_id) will itself contain two elements: the first (acc_id[0]) is a copy of an item from the original list (blocked_accounts.results); the second (acc_id[1]) is the subelement from acc_id[0] that this pair corresponds to.

It’s easy to get lost in all the words, but the simple demo is this:

  orig:
    - alpha: a
      numbers:
        - 1
        - 2
    - alpha: b
      numbers:
        - 3
        - 4
  result: "{{ orig | subelements('numbers') }}"
# result looks like this:
    - - alpha: a
        numbers:
          - **1**
          - 2
      - **1**
    - - alpha: a
        numbers:
          - 1
          - **2**
      - **2**
    - - alpha: b
        numbers:
          - **3**
          - 4
      - **3**
    - - alpha: b
        numbers:
          - 3
          - **4**
      - **4**
6

Thanks much Todd, the above code works as expected.