with_dict expects a dict...can a dict be too large

Archives Ansible Project
1

I messed up and deleted my first post. Sorry!

I think I need another pair of eyes here. I am attempting to run a dict of users through an ipa user-add command, but it fails with the error “with_dict expects a dict.” I believe I have formatted my dict correctly (it’s about 40,000 lines long, generated from a python3 script):

---
students:
username1:
lastname: jibberish
firstname: jibberish
firstinitial: jibberish
studentid: jibberish
dob: jibberish
dateexpires: 2018-06-19
epochexpiry: 1529415778
uid: jibberish
passwordhash: "jibberish"
username2:
lastname: jibberish
firstname: jibberish
firstinitial: jibberish
studentid: jibberish
dob: jibberish
dateexpires: 2018-06-19
epochexpiry: 1529416593
uid: jibberish
passwordhash: "jibberish"
username3:
lastname: jibberish
firstname: jibberish
firstinitial: jibberish
studentid: jibberish
dob: jibberish
dateexpires: 2018-06-19
epochexpiry: 1529416755
uid: jibberish
passwordhash: "jibberish"

And here is the play where I am trying to use it:

`

  • name: Add existing student users to ipaserver
    command: “/bin/ipa user-add
    {{ item.key }}
    –first=‘{{ item.value.firstname }}’
    –last=‘{{ item.value.lastname }}’
    –cn=‘{{item.value.firstname}} {{item.value.lastname}}’
    –email={{ item.value.name }}@domain
    –shell=/bin/bash
    –homedir=‘/home/students/{{ item.value.firstinitial }}/{{ item.key }}’
    –uid=‘{{ item.value.uid }}’
    –gidnumber=‘{{ item.value.uid }}’
    –skeleton=/etc/skel
    –setattr userpassword=‘{crypt}{{ item.value.passwordhash}}’
    –title=student”

with_dict: “{{students}}”
register: added_students

`

And the error:

TASK [Add existing student users to ipaserver] ********************************* fatal: [216.125.253.133]: FAILED! => {"failed": true, "msg": "with_dict expects a dict"}
Running it verbose only gives me the task path in addition to the above error.

Is there a maximum size a dict can be? Is there something else wrong with my dict? I have tried it as a list (with a - before each username and with_items in the play), but got a unicode error. I’m stumped.

Thanks for taking a look!

2

I always find with_dict fiddly, it'll work eventually. I can't see
anything obvious at first glance
(
I do think

   --email={{ item.value.name }}@domain

should be

   --email="{{ item.key }}@domain"

but I don't think Ansible is getting as far as even opening the dict yet
)

When I get this kind of issue, I cut down the size of the dict to a
little sample set, just to verify the YAML
is formatted correctly (no whitespace gotchas?).

Also, are you sure the dict is in scope at this point?

3

Hi Dick,

Thanks for replying! Good catch on the email address. I’ll tweak that.
I tried even just a debug of printing each item and that had the same with_dict error.

It’s a good idea to try a truncated dict; then at least I’ll have a better idea whether it’s the length breaking it. That’s my next step.

Thanks!
Joanna

4

Good question about whether the dict is in scope.
I import the vars this way at the top of the playbook:

`

vars_files:

  • vars/studentdict.yml
  • vars/staffdict.yml

`

I run the playbook with become: yes. I verified that root and the local ansible user can read the dict.

I wonder if dashes in my date fields are breaking it. I am testing quoting those entries right now.

5

I don’t have an exact answer as to why currently, but the problem is with your dateexpires values. That value gets converted to a python datetime object. And because of it being automatically parsed as a datetime object, the dict is transformed into a string representation of the dict. The conversion to a string representation is the part that I don’t fully understand right now. It seems to be something inside of jinja2 itself that is causing this behavior.

So effectively you end up with a list that has 1 element, and that 1 element is a string representation of your dict.

Simply quoting the value of dateexpires resolves the issue, such as:

username1:
lastname: jibberish
firstname: jibberish
firstinitial: jibberish
studentid: jibberish
dob: jibberish
dateexpires: “2018-06-19”
epochexpiry: 1529415778
uid: jibberish
passwordhash: “jibberish”

6

Yup, a test with a short dict of only three records failed. Until I quoted all the date fields containing dashes.
Each user record contains two dates that formatted like YYYY-MM-DD, which apparently breaks yaml. It makes sense now, but hadn’t occurred to me until I was driving home a while ago.

I was using this:
dob: 1962-01-21 dateexpires: 2018-06-19 epochexpiry: 1529415778

Dashes break hashes, so it fixed it to quote the values:

dob: '1962-01-21' dateexpires: '2018-06-19' epochexpiry: 1529415778

I have yet to tweak my 40,000 lines (about 4000 records) and try with those.

7

It turns out I had various dashes and apostrophes (isolated single quotes spell trouble, too) smattered throughout the user’s first and last names. So, after quoting dates, password hashes, and name fields, I tried with the full large dictionary. Now my play works! Even more exciting, I have new users appearing on my IPA server with appropriate data. Thanks Dick and Matt!

If you’re ever in Champaign, IL USA, look me up. I’ll buy you a coffee.

Joanna