Type 'GroupTuple' is unsupported in variable storage

Get Help
1

Given the list

data:
  - {group: devel, team: dev_1, users: [alice, bob]}
  - {group: devel, team: dev_2, users: [charlie, david]}
  - {group: qa, team: qa_1, users: [mallory, ted]}

The Jinja filter groupby works as expected. The below debug task

    - debug:
        msg: "{{ data | groupby('group') }}"

however, shows this warning

TASK [debug] ********************************************************************************************************
[WARNING]: Type 'GroupTuple' is unsupported in variable storage, converting to 'list'.
Origin: /export/scratch/sandbox/tmp1/test-187/pb6.yml:13:14

11
12     - debug:
13         msg: "{{ data | groupby('group') }}"
                ^ column 14

ok: [localhost] => 
    msg:
    -   - devel
        -   -   group: devel
                team: dev_1
                users:
                - alice
                - bob
            -   group: devel
                team: dev_2
                users:
                - charlie
                - david
    -   - qa
        -   -   group: qa
                team: qa_1
                users:
                - mallory
                - ted

Explicit conversion doesn’t help

    - debug:                                                                                                         
        msg: "{{ data | groupby('group') | list }}"

I don’t understand. The below debug

    - debug:
        msg: |
          {{ data | type_debug }}
          {{ result | type_debug }}
          {{ result }}
      vars:
        result: "{{ data | groupby('group') }}"

shows no warnings. But, I see the warning again if the result is used in a loop

    - debug:
        var: item
      loop: "{{ result }}"
      vars:
        result: "{{ data | groupby('group') }}"

Is this a bug?

1 Like
2

The first thing I’d try is to use a key name other than “group” since that could be a reserved keyword.

3

Update this to:

    - debug:
        msg: "{{ data | groupby('group')|map('list') }}"

That is what is happening under the hood anyway, when the GroupTuple is found. This basically requires an explicit action, or that is what the warning is wanting you to do, because this is a “lossy” operation, which loses information in the “namedtuple” that jinja2 is returning.

The reason for this is that there is no standard JSON serialization for a namedtuple, it requires making a decision about this, and in this case, it’s probably ok for your use to just explicitly convert the inner items to a list.

2 Likes
4

Thank you. Do I have to explicitly convert all such filters to eliminate these particular warnings? Is the user now responsible for handling this?

This situation is weird. This debug

    - debug:
        msg: |
          {{ result }}
      vars:
        result: "{{ data | groupby('group') }}"

converts the tuples to lists and shows the warning

TASK [debug] ********************************************************************************************************
[WARNING]: Type 'GroupTuple' is unsupported in variable storage, converting to 'list'.
Origin: /export/scratch/sandbox/tmp1/test-187/pb8.yml:23:14

21
22     - debug:
23         msg: |
                ^ column 14

ok: [localhost] => 
    msg:
    -   - devel
        -   -   group: devel
                team: dev_1
                users:
                - alice
                - bob
            -   group: devel
                team: dev_2
                users:
                - charlie
                - david
    -   - qa
        -   -   group: qa
                team: qa_1
                users:
                - mallory
                - ted

But, adding a trivial line

    - debug:
        msg: |
          {{ inventory_hostname }}
          {{ result }}
      vars:
        result: "{{ data | groupby('group') }}"

makes the block happy with GroupTuple

TASK [debug] ********************************************************************************************************
ok: [localhost] => 
    msg: |-
        localhost
        [('devel', [{'group': 'devel', 'team': 'dev_1', 'users': ['alice', 'bob']}, {'group': 'devel', 'team': 'dev_2', 'users': ['charlie', 'david']}]), ('qa', [{'group': 'qa', 'team': 'qa_1', 'users': ['mallory', 'ted']}])]

Such blocks must be identified in 2.19 porting. Are there any other surprises next to groupby?

5

You probably refer to Referencing key-value dictionary variables. This is not the case here. JFYI, YAML places no restrictions on the type of keys.

6

Thanks. It was an off the cuff “first thing I thought of” hit to rule things out. Glad you found the right answer.

7

Yes, there are. The below play

- hosts: foo

  tasks:

    - command: zpool status
      register: out

    - debug:
        var: pools
      vars:
        status: "{{ out.stdout | community.general.jc('zpool-status') }}"
        pools: "{{ dict(status | community.general.json_query('[].[pool, config[].name]')) }}"

gives, for example

    pools:
        iocage:
        - iocage
        - ada2
        - ada3
        zroot:
        - zroot
        - ada0p4
        - ada1p4

However, ansible-lint complains about

pools: "{{ dict(status | community.general.json_query('[].[pool, config[].name]')) }}"

jinja[invalid]: Error rendering template: ‘NoneType’ object is not iterable

Mapping the items from the json_query output list to list makes the lint error disappear

pools: "{{ dict(status | community.general.json_query('[].[pool, config[].name]') | map('list')) }}"

I have to repeat my questions:

Is this a bug?

Do I have to explicitly convert all such filters to eliminate these particular warnings (and errors)?

Is the user now responsible for handling this?

1 Like
8

There isn’t a single answer. The groupby filter is coming from upstream jinja2, and json_query is in community.general.

We don’t have knowledge of every python type returned by every filter out there. It’s dependent on the filter, and the reason there is a warning, is to help try to let you know there is something that needs done.

Unfortuantely we can’t know every situation, and these type of things will have to be handled on a per case basis, and using list is only 1 solution, for list type values. Other types may be required elsewhere, and I cannot enumerate where they may happen.

2 Likes
9

For the record (to clarify the solution):

Is this a bug? No

Do I have to explicitly convert all such filters to eliminate these particular warnings (and errors)? Yes

Is the user now responsible for handling this? Yes

1 Like
10

For the case @vbotka cited above, could something be done inside or with community.general.json_query() to make ansible-lint do the Right Thing™? A lot of people use json_query, and fixing a class of linting issues would be far preferable to everyone squashing those linting issues in inconsistent ways…

1 Like
11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.