module params doing JSON conversion?

Archives Ansible Developer
1

Hi all,

I’ve got a JSON file, web_iam_trust_policy.json:

{
    "Version": "2012-10-17",
    "Statement":
    [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal":
            {
                "Service": "ec2.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

I’m trying to use it in a lookup:

        trust_policy: "{{ lookup('file', 'files/aws/web_iam_trust_policy.json') }}"

However, when the param is loaded it’s being converted into a Python dict and then into a string. Within the module:

    trust_policy=dict(type='str’),
    ...
    trust_policy = module.params['trust_policy']
    print ‘PARROT’, type(trust_policy), trust_policy

Prints:

PARROT <type ‘str’> u'Version': u'2012-10-17', u'Statement': [{u'Action': u'sts:AssumeRole', u'Sid': u'', u'Effect': u'Allow', u'Principal': {u'Service': u'ec2.amazonaws.com'}}]}

If I leave off the type=‘str’, the file is loaded as a dict:

    trust_policy=dict(),
    ...
    trust_policy = module.params['trust_policy']
    module.exit_json(changed=True, tp=trust_policy)

changed: [127.0.0.1] => {"changed": true, "tp": {"Statement": [{"Action": "sts:AssumeRole", "Effect": "Allow", "Principal": {"Service": "ec2.amazonaws.com"}, "Sid": ""}], "Version": "2012-10-17"}}

How can I prevent this conversion? I just want the raw file contents. debug msg=“{{ lookup(..) }}” does not display this behavior.

Regards,
-scott

2

“However, when the param is loaded it’s being converted into a Python dict and then into a string. Within the module”

Nope, the file lookup plugin just returns strings.

You’d need to pass this to the “from_json” filter most likely, but you know what’s an even better way to do this?

vars_files or include_vars!

This is exactly what they do, and they can also take JSON inputs.

3

Yes, it’s a string when lookup lets go of it. However, once it’s a module param it’s converted from JSON into a dict. Notice this line:

Lookup has loaded it as a string, but when I pull it out of the module params it has been 1) parsed as JSON and then 2) converted back into a string. See the u’…’ keys and values? That’s a JSON string that has been run through loads (converting it to a dict) and then back into a string representation of a dict.

I don’t want it as a dict or vars at all, just a raw string that happens to have JSON in it. I’m passing it to boto as a policy file. vars or include_vars is exactly what I don’t want. :slight_smile:

Regards,
-scott

4

Note that the ec2_lc module has the same issue with the user_data field. When I was using it to put a shell script in user_data it worked fine. Now that I’ve changed to using JSON, I get this in user_data instead of the raw JSON that I need:

{u’app_environment’: u’stage’, u’database_name’: u’sp_app’, u’database_endpoint_port’: u’3306’, u’database_endpoint_host’: u’sp-stage-rds-instance-master.parrot.us-west-1.rds.amazonaws.com’}

  • name: Create new launch configuration
    local_action:
    module: ec2_lc
    name: “{{ environment_web_launch_configuration_name }}”
    state: present
    user_data: “{{ lookup(‘template’, ‘…/templates/user_data.json’) }}”
    […]

This is happening after the lookup runs, but before the module extracts the parameters.

Regards,
-scott

5 

Doesn't seem like it to me:

From ec2_lc:

            user_data=dict(type='str'),

There is nothing in the module code that coerces the type.

Some fields do coerce inputs, like the tags field on the ec2 module. This one does not.

6

Here’s a test case.

The problem is only displayed in actions (or local_actions, not shown below but I did test it) with JSON in parameters that have variable substitutions in them. There’s definitely a different path being followed given the REMOTE_MODULE lines are different.

test_params module:

#!/usr/bin/python

def main():
module = AnsibleModule(
argument_spec=dict(
thing=dict(type=‘str’),
)
)

thing = module.params.get(‘thing’)

module.exit_json(changed=False, thing=thing)

import module snippets

from ansible.module_utils.basic import *

main()

pb_test.yml:

7

Great! Very useful.

Can you please file this as a bug report so we don’t lose these steps.

Thanks!!!

8

Hi,

We stumbled upon the same bug: you cannot form a valid JSON as user_data using ec2_lc module. It sets the single quotes instead of double ones :frowning:

We have to encode user_data to Base64 to work around the bug. Probably it can be helpful to others.

Kind regards,
Vyacheslav.

9

Was a bug ever filed?

I am coming across the same bug. All I need is to read the raw contents of a file and that just happens to have json in it. But AnsibleModule has the read the content as dict and hence I am unable to get the raw string.