Splitting a variable length list into a list of lists

Archives Ansible Project
1

Greetings,

I’d like to take a list (of varying size) and split it into a list of lists, where each sub-list is at most N elements. I’m not certain where to start on this one. Advice welcome.

The background is that I’ve got a dynamically built list of packages for installation. For Centos 6.x, if the list is too large, the yum module will fail out, reporting “ValueError: filedescriptor out of range in select()”. I’d like to split the large list into sub-lists, and process each sub-list with separate yum tasks. I’m not clear how I can use Jinja / Ansible to build that list of lists. I figure if I can craft the list of lists, I can use with_items on an include, where the include file will run the yum install on one sub-list at a time.

I’m open to other approaches on how to solve the problem.

Thanks,
Ed

2

Are you sure the size of the list is the source of your error? It sounds more like an “open files” issue. Whats the output of ulimit -n?

3

It’s a combination of both, actually. I can boost the nofile setting, but I’m still able to construct a large enough package list that it fails. Things work fine in Centos 7.x - I don’t know if the resolution arrived in python or in Centos.

I’m going to try using the batch filter for my sub-lists. I’d somehow missed it in my searching.

4

I think I have a working solution. The outer file has:

  • set_fact:
    batches: “{{ packages|batch(batch_size)|list }}”

  • include: run_installs.yml
    with_list: “{{ batches }}”
    loop_control:
    loop_var: batch

Then run_installs.yml is simply:

  • name: Update and install
    yum:
    state: latest
    name: “{{ item }}”
    with_items: “{{ batch }}”
    become: true
    become_user: root

I have to use with_list in the outer file since with_items will flatten lists (ref: https://stackoverflow.com/questions/43902117/ansible-list-of-lists-flattening).

Ed