I had another topic about using a fact in a loop, but since this question is slightly different, I figured I’d start a new topic.
Background:
I’m using the IPA API calls using the uri module to get a list of users in our IPA server. Once I get the uses via user_find, I set a fact of just the uid’s. This works well. Then I am using that fact to use the user_show API via uri. This also works, but since we have over 1600 users, it takes a bit of time. So I’m looping through the uid list to process the first 20.
Issue:
I am registering the user_show data in a variable called user_show. I’m trying to set a fact that has the uid, password expiration, and the age calculation. The problem I have is that the fact has the information from the last user processed, and is not an array of data as I expect. Here’s how I’m setting the facts:
name: Run user_show from IDM API using previously stored session cookie
uri:
url: “https://{{idmfqdn}}/ipa/session/json”
method: POST
headers:
Cookie: “{{ login.set_cookie }}”
Referer: “https://{{idmfqdn}}/ipa”
Content-Type: “application/json”
Accept: “application/json”
body_format: json
body: “{"method": "user_show","params": [[ "{{ uid[item|int] }}"],{"all": true,"version": "{{ api_vers }}"}]}”
register: user_show
with_sequence: start=0 end=19
The problem with implementing it that way is that the users’s fact is referencing "user_show.results[item|int], but I can’t add another loop variable. I’m using with_sequence earlier in the playbook for testing to limit the amount of users queried, and that number is needed for the user_show.results array.
The problem with implementing it that way is that the users’s fact is referencing "user_show.results[item|int], but I can’t add another loop variable. I’m using with_sequence earlier in the playbook for testing to limit the amount of users queried, and that number is needed for the user_show.results array.
Hi,
I wanted to know why you are using item to retrieve results - “{{ user_show.results[item|int].json.result.result.uid[0] }}” . since item is useful when you have a loop running. Is it not possible to retrieve the result using json_query or the map filter ?If that can be done , then we won’t need to bring in the loop_control variable.From my experience, its better to get the basic stuff (with_X)to work first before using the latest stuff- loop/zip (filter).
For my testing, I’m calling the user_show IPA API via uri for the first 5 items. The registered variable called “user_show” has the json value of “user_show.results.json.result.result.Y” (where X is 0 for the first user, 1 for the second, etc; and Y is the user property (uid, firstname, etc.). I don’t know how to get/retrieve just the uid property of those 5 users, or just the krbpasswordexpiration of those 5 users. I don’t have to do a loop, but nothing I try seems to let me get past the fact that the results json is an array, and I need to reference the particular items of that array.
I’m including the playbook below, followed by the relevant output. I have redacted the real data values with dummy values, but it should give you an idea of what I’m trying and what I’m getting.
Hi there,
I created this sample playbook and based on the data you sent.
I used the mail field because that was not embedded in another key/value pair like krbpasswordexpiration but the principle is basically the same.
The difference with your playbook is that I used a variable to register the set_fact . Without that, it would just show the value of the last item in the loop which will overwrite the previous values. By registering the output , I can extract out all the values in the last task.
There are probably other methods but this one works.
I finally installed FreeIPA in a VM on my laptop so I could see what you’re seeing. Except for having a self-signed certificate and way less than 1500 users, I think I’ve got something I can test on. A few things to note, though they may not be all that you’re looking for:
Exclude your ‘admin’ user. It shouldn’t be an issue, but, just in case.
I used loop: “{{ uid | json_query(‘[:20]’) }}” to limit the ‘user_show’ calls rather than with_sequence:. Not better or worse, just showing another way to do it. Since we’re both using json_query later, it has to be available anyway.
Once you’ve got your show_user output, you can go straight to a list of users to delete - with their pw expiration dates - in one json_query. There’s no need to create so many intermediate set_fact lists.
I didn’t notice before, but you aren’t doing nested loops after all, so all that stuff I mentioned about loop_control is irrelevant. Sorry about the rabbit hole.
Here’s my current playbook to produce a list like
uid: user1
pwdepx: 20201210012032Z
uid: user2
pwdepx: 20181080021425Z
…
of just the users who are eligible for deletion. Hope this helps.
name: Set fact for users
set_fact:
uid: “{{ user_find.json.result.result|map(attribute=‘uid’)|flatten|difference([‘admin’]) }}”
name: Run user_show from IDM API using previously stored session cookie
uri:
url: “https://{{idmfqdn}}/ipa/session/json”
validate_certs: “{{ validate_certs }}”
method: POST
headers:
Cookie: “{{ login.set_cookie }}”
Referer: “https://{{idmfqdn}}/ipa”
Content-Type: “application/json”
Accept: “application/json”
body_format: json
body:
method: user_show
params:
name: Save “{{ cutoff_days }} days ago” in krbpasswordexpiration.datetime format
set_fact:
cutoff_date: ‘{{ lookup(’‘pipe’‘,
‘‘date -u --date=“{{ cutoff_days }} days ago” +%Y%m%d000000Z’’) }}’
name: Set deletable_users fact
set_fact:
deletable_users: “{{ user_show.results |
json_query(‘[*].json.result.result.{uid: uid[0], pwdexp: krbpasswordexpiration[0].datetime}’) |
selectattr(‘pwdexp’,‘<’,cutoff_date) | list }}”
OK, I got that to work following your example. Thanks for that. So the next step I’m trying to get to is to determine if the user’s password has been expired for more than 180 days. A negative value means its not expired, and a positive value means that it is expired. I know you used email, but I’m using password expiration (krbpasswordexpiration), and I am actually calculating the password age in the fact at the same time. So now the playbook looks like this:
name: Grab UID/Password Expiration/Disabled Status from user_show
set_fact:
users: “{{ user_show.results|json_query(‘[].json.result.result.uid[0]‘)|flatten }}"
pwdexp: "{{ user_show.results|json_query(’[].json.result.result.krbpasswordexpiration[0]’)|flatten }}”
name: Combine data into one fact
set_fact:
tmpuserlist1: “{{ item.0 }} {{ (ansible_date_time.epoch|int - ((item.1[‘datetime’] | to_datetime(‘%Y%m%d%H%M%SZ’)).strftime(‘%s’))|int) / (606024) }}”
with_together:
“{{ users }}”
“{{ pwdexp }}”
register: tmpuserlist2
name: Set Temporary User List fact
set_fact:
userlist: “{{ tmpuserlist2.results|map(attribute=‘ansible_facts.tmpuserlist1’)|list }}”
name: Print Final User List
debug:
msg: “{{ userlist }}”
Which gives the following:
TASK [Grab UID/Password Expiration/Disabled Status from user_show] ****************************************************************
ok: [ipaserver.example.com]
TASK [Combine data into one fact] *************************************************************************************************
ok: [ipaserver.example.com] => (item=[u’test.user1’, {u’datetime’: u’20220103195934Z’}])
ok: [ipaserver.example.com] => (item=[u’test.user2’, {u’datetime’: u’20200218151047Z’}])
TASK [Set Temporary User List fact] ***********************************************************************************************
ok: [ipaserver.example.com]
TASK [Print Final User List] ******************************************************************************************************
ok: [ipaserver.example.com] => {
“msg”: [
“test.user1 -24.2161574074”,
“test.user2 660.984386574”
]
}
I think that the next step should be to create a list of users who’s age is greater than or equal to 180, then go loop through that list and disable them. But I can’t seem to figure out how to grab the age field now that it’s in the new “userlist” fact. If I try to show each item individually via a loop like this:
name: Print Final User List
debug:
msg: “{{ item }}”
loop: “{{ userlist }}”
So a few things that I tried based on Uto’s suggestion: I changed the cutoff days to 30 for testing, and I only look over the first 10 users. I had to change the < check to >, but when the disabled_users fact is printed, It shows the same user all 10 times, almost as if it added the user 10 times. Any ideas?
Harry
name: Run user_show from IDM API using previously stored session cookie
uri:
url: “https://{{idmfqdn}}/ipa/session/json”
method: POST
headers:
Cookie: “{{ login.set_cookie }}”
Referer: “https://{{idmfqdn}}/ipa”
Content-Type: “application/json”
Accept: “application/json”
body_format: json
body: “{"method": "user_show","params": [[ "{{ uid[item|int] }}"],{"all": true,"version": "{{ api_vers }}"}]}”
register: user_show
loop: “{{ uid|json_query(‘[:10]’) }}”
name: Save “{{ cutoff_days }}” in Password Expiration format
set_fact:
cutoff_date: ‘{{ lookup(’‘pipe’‘, ‘‘date -u --date=“{{ cutoff_days }} days ago” +%Y%m%d000000Z’’) }}’
So a few things that I tried based on Uto's suggestion: I changed the cutoff days to 30 for testing, and I only look over the first 10 users. I had to change the < check to >, but when the disabled_users fact is printed, It shows the same user all 10 times, almost as if it added the user 10 times. Any ideas?
That is strange. I'll go create some more test users and back-date their pw expiration dates. Hmm.
When you change from with_sequence to loop: “{{ uid|json_query(‘[:10]’) }}”, item is no longer an index into the uid list. It’s actual uids. So your body becomes
body:
method: user_show
params:
(And you don’t have to generate the json string yourself. The module will do that for you. Much easier to read/maintain than
body: “{"method": "user_show","params": [[ "{{ item }}"],{"all": true,"version": "{{ api_vers }}"}]}”
but I fixed that line just in case you like that better.)
Also, change the ‘>’ back to ‘<’. Earlier date strings are ‘<’ later date strings.
Thank you a MILLION! Got it to work, and its much simpler too. I can’t thank you enough for going above and beyond the call in helping me out. I’ll keep working on it next week, but for now its doing exactly what I need it to do at this point. And I picked up a few tricks along the way, like the json_query(‘[:20]’) and how to use the difference filter to ignore user names I don’t need to check. Great stuff here!
I have another quick question: I got this to work and it currently shows ALL disabled accounts. I can also grab the nsaccountlock value, which will show True if the account is already disabled and False if it is not. What’d I’d like to do is filter the results to users who’s password has been expired 180 days or longer (selectattr(‘pwdexp’, “<”,cutoff_date) AND nsaccountlock is false. How do I specify that in the example you gave using selectattr? My “disabled users” fact is currently:
Hi there,
Have you tried using selectattr(‘nsaccountlock’, ‘true’) if the value is logical (True / false) as an extra filter in the set_fact task?
Also a long route could be to compare deletable_users with disabled_users and use the intersection filter
{{ deletable_users | intersect(disabled_users) }}
