Loop through external API with multiple offsets

Archives AWX Project
1

Hello Folks,

I have a requirement to loop through an REST API, to get data, and I would like to use ANSIBLE YAML.

The API itself only renders 400 entries(limit) at a time and hence to get the entire data set will need to loop through all offsets incrementing every time by 400.

Each time the api is called the resulting JSON has a key which tells whether more results are available or not.

The non-trivial piece for me:

  1. The end count is unknown, hence cannot use loop or with_items/stride etc.
  2. The condition to goto next increment is part of the resulting JSON from API get.

I was able to make this work via python but want to use available URI module to do the same.

Any ideas are much appreciated!

thanks
drake

2

You can trivially figure out the end count on your own, because you know the page size and the API reports the total count. However, doing a static loop has a self-obvious contradiction that you can’t know how many loops you need to do until you get the first response back.

Iterators are a more elegant way of doing this, but it’s also sufficient to do a python while loop while the next link is not null. tower-cli as an all_pages parameter that you can borrow the implementation from. Or you could share your code and get suggestions.