# Trouble parsing JSON data returned from URI call - making use of user-data in EC2 **URL:** \ **Category:** Ansible Project\ **Created:** [March 14, 2014, 12:35am UTC](https://forum.ansible.com/t/trouble-parsing-json-data-returned-from-uri-call-making-use-of-user-data-in-ec2/14769 "2014-03-14T00:35:16Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Ben\_Turner](https://avatars.discourse-cdn.com/v4/letter/b/77aa72/32.png) [@Ben\_Turner](https://forum.ansible.com/u/Ben_Turner)\ **Post date:** [March 14, 2014, 12:35am UTC](https://forum.ansible.com/t/trouble-parsing-json-data-returned-from-uri-call-making-use-of-user-data-in-ec2/14769/1 "2014-03-14T00:35:16Z")
I’ve been trying to use the user-data provided by EC2 servers to determine a servers hostname. So currently new server come up with a hostname of “ec2-123-234-345-456” or similar, and I’d like in my ansible script to rename that server to something like “demonstration”. To do this, I’ve gone into my EC2 console, and added the following as user-data to my server: {“hostname”: “demonstration”} Then, in my ansible playbook, I’ve tried the following: - name: Load user data uri: url=[http://169.254.169.254/latest/user-data](http://169.254.169.254/latest/user-data) return\_content=yes register: user\_data - debug: var=user\_data.content - name: Set hostname from user data command: hostname {{user\_data.content.hostname}} when: user\_data is defined This produces the output (using the -v flag): PLAY [ec2] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* GATHERING FACTS \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* ok: [ec2-54-253-114-###.ap-southeast-2.compute.amazonaws.com] TASK: [common | Load user data] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* ok: [ec2-54-253-114-###.ap-southeast-2.compute.amazonaws.com] =\> {“accept\_ranges”: “bytes”, “changed”: false, “connection”: “close”, “content”: “{"hostname":"demonstration"}”, “content\_length”: “28”, “content\_location”: “[http://169.254.169.254/latest/user-data](http://169.254.169.254/latest/user-data)”, “content\_type”: “application/octet-stream”, “date”: “Fri, 14 Mar 2014 00:30:30 GMT”, “etag”: “"1016948274"”, “item”: “”, “last\_modified”: “Thu, 13 Mar 2014 22:02:44 GMT”, “redirected”: false, “server”: “EC2ws”, “status”: 200} TASK: [common | debug var=user\_data.content] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* ok: [ec2-54-253-114-###.ap-southeast-2.compute.amazonaws.com] =\> { “item”: “”, “user\_data.content”: { “hostname”: “demonstration” } } TASK: [common | Set hostname from user data] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* fatal: [ec2-54-253-114-###.ap-southeast-2.compute.amazonaws.com] =\> One or more undefined variables: ‘unicode object’ has no attribute ‘hostname’ So, whilst it seems that debugging user\_data returns a JSON-like return strucutre, and drilling down to user\_data.content also returns a JSON-like return structure, the actual JSON content returned by the uri task is unable to be parsed using the Jinja2 dot notation, and instead appears to be an impenetrable object, at least as far as I can determine. Is there a way of extracting this URI return content using core ansible modules such that I can use it’s JSON payload as ansible variables ? Regards, Ben ---
**Author:** ![Romeo\_Theriault](https://avatars.discourse-cdn.com/v4/letter/r/839c29/32.png) [@Romeo\_Theriault](https://forum.ansible.com/u/Romeo_Theriault)\ **Post date:** [March 14, 2014, 1:15am UTC](https://forum.ansible.com/t/trouble-parsing-json-data-returned-from-uri-call-making-use-of-user-data-in-ec2/14769/2 "2014-03-14T01:15:47Z")
Hi Ben, If you want to access the JSON data it's available in the json subkey. So instead of: {{ user\_data.content.hostname }} I believe it would be {{ user\_data.json.hostname }} or something similar. Take a look at the {{ user\_data.json }} or {{ user\_data }} output with the debug module to get a better idea. Hope that helps, Romeo ---
**Author:** ![Ben\_Turner1](https://avatars.discourse-cdn.com/v4/letter/b/57b2e6/32.png) [@Ben\_Turner1](https://forum.ansible.com/u/Ben_Turner1)\ **Post date:** [March 14, 2014, 2:27am UTC](https://forum.ansible.com/t/trouble-parsing-json-data-returned-from-uri-call-making-use-of-user-data-in-ec2/14769/3 "2014-03-14T02:27:53Z")
I tried out some more debug statements; the .json suffix doesn’t appear to be present ? The user-data “json” appears to be coming back as a string… TASK: [common | debug var=user\_data] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* ok: [[ec2-54-253-114-32.ap-southeast-2.compute.amazonaws.com](http://ec2-54-253-114-32.ap-southeast-2.compute.amazonaws.com)] =\> { “item”: “”, “user\_data”: { “accept\_ranges”: “bytes”, “changed”: false, “connection”: “close”, “content”: “{"hostname":"qa"}”, “content\_length”: “17”, “content\_location”: “[http://169.254.169.254/latest/user-data](http://169.254.169.254/latest/user-data)”, “content\_type”: “application/octet-stream”, “date”: “Fri, 14 Mar 2014 02:26:12 GMT”, “etag”: “"1016948274"”, “invocation”: { “module\_args”: “url=[http://169.254.169.254/latest/user-data](http://169.254.169.254/latest/user-data) return\_content=yes”, “module\_name”: “uri” }, “item”: “”, “last\_modified”: “Thu, 13 Mar 2014 22:02:44 GMT”, “redirected”: false, “server”: “EC2ws”, “status”: 200 } } TASK: [common | debug var=user\_data.json] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* ok: [[ec2-54-253-114-32.ap-southeast-2.compute.amazonaws.com](http://ec2-54-253-114-32.ap-southeast-2.compute.amazonaws.com)] =\> { “item”: “”, “user\_data.json”: “{{ user\_data.json }}” } TASK: [common | debug var=user\_data.content.json] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* ok: [[ec2-54-253-114-32.ap-southeast-2.compute.amazonaws.com](http://ec2-54-253-114-32.ap-southeast-2.compute.amazonaws.com)] =\> { “item”: “”, “user\_data.content.json”: “{{ user\_data.content.json }}” } ---
**Author:** ![Michael\_DeHaan1](https://avatars.discourse-cdn.com/v4/letter/m/5e9695/32.png) [@Michael\_DeHaan1](https://forum.ansible.com/u/Michael_DeHaan1)\ **Post date:** [March 14, 2014, 2:39pm UTC](https://forum.ansible.com/t/trouble-parsing-json-data-returned-from-uri-call-making-use-of-user-data-in-ec2/14769/4 "2014-03-14T14:39:09Z")
In your Debug output it looks like it came out that way because that was what got stored there. ---
**Author:** ![Ben\_Turner1](https://avatars.discourse-cdn.com/v4/letter/b/57b2e6/32.png) [@Ben\_Turner1](https://forum.ansible.com/u/Ben_Turner1)\ **Post date:** [March 17, 2014, 9:57am UTC](https://forum.ansible.com/t/trouble-parsing-json-data-returned-from-uri-call-making-use-of-user-data-in-ec2/14769/5 "2014-03-17T09:57:24Z")
Looks like all “user-data” in AWS is just a “string” at the end of the day. So there is no way for Ansible to parse that string and pull out values, without a third-party module such as [https://github.com/jpmens/ansible-ec2-userdata](https://github.com/jpmens/ansible-ec2-userdata) say ? Ben ---
**Author:** ![sivel](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sivel/32/1198_2.png) [@sivel](https://forum.ansible.com/u/sivel)\ **Post date:** [March 17, 2014, 11:28am UTC](https://forum.ansible.com/t/trouble-parsing-json-data-returned-from-uri-call-making-use-of-user-data-in-ec2/14769/6 "2014-03-17T11:28:50Z")
Doesn’t the ec2\_facts module already query user-data and parse into usable data? I was pretty sure it did. ---
**Author:** ![Dag\_Wieers](https://avatars.discourse-cdn.com/v4/letter/d/45deac/32.png) [@Dag\_Wieers](https://forum.ansible.com/u/Dag_Wieers)\ **Post date:** [March 17, 2014, 11:49am UTC](https://forum.ansible.com/t/trouble-parsing-json-data-returned-from-uri-call-making-use-of-user-data-in-ec2/14769/7 "2014-03-17T11:49:56Z")
It would be a very simple improvement to set\_fact to allow a string to become a json-object, however the syntax of set\_fact was made so that every parameter is a fact name. However a set\_fact\_from\_json module would be super-easy to implement, so that you can combine it with uri (or other registered variables). Another option is to extend the uri module with an option like: content\_as\_json=yes My prefered solution is to include this in the inventory step (outside of Ansible) so that it can be processed asynchronous (and cached) to Ansible. ---
**Author:** ![sivel](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sivel/32/1198_2.png) [@sivel](https://forum.ansible.com/u/sivel)\ **Post date:** [March 17, 2014, 12:30pm UTC](https://forum.ansible.com/t/trouble-parsing-json-data-returned-from-uri-call-making-use-of-user-data-in-ec2/14769/8 "2014-03-17T12:30:48Z")
It is actually possible to use set\_fact with json data already, or to use info from json data for pretty much anything. You just need to use the from\_json filter. - set\_fact: hostname: “{{ (user\_data.content|from\_json).hostname }}” or in the example given: - name: Set hostname from user data command: “hostname {{ (user\_data.content|from\_json).hostname }}” when: user\_data is defined Although 2 things to mention: 1. As mentioned, I do think that the ec2\_facts module already grabs user-data \<[https://github.com/ansible/ansible/blob/devel/library/cloud/ec2\_facts#L61](https://github.com/ansible/ansible/blob/devel/library/cloud/ec2_facts#L61)\> If I am reading the code correctly, would end up looking like: {{ hostvars[inventory\_hostname][‘ansible\_ec2\_user-data’][‘hostname’] }} 1. Instead of issuing a hostname change via the command module, look at the hostname module \<[http://docs.ansible.com/hostname\_module.html](http://docs.ansible.com/hostname_module.html)\> ---
**Author:** ![Dag\_Wieers](https://avatars.discourse-cdn.com/v4/letter/d/45deac/32.png) [@Dag\_Wieers](https://forum.ansible.com/u/Dag_Wieers)\ **Post date:** [March 17, 2014, 12:38pm UTC](https://forum.ansible.com/t/trouble-parsing-json-data-returned-from-uri-call-making-use-of-user-data-in-ec2/14769/9 "2014-03-17T12:38:44Z")
Right, didn't think of doing it like this. Quite powerful indeed. Thanks 🙂 ---
**Author:** ![Ben\_Turner](https://avatars.discourse-cdn.com/v4/letter/b/77aa72/32.png) [@Ben\_Turner](https://forum.ansible.com/u/Ben_Turner)\ **Post date:** [March 24, 2014, 2:58am UTC](https://forum.ansible.com/t/trouble-parsing-json-data-returned-from-uri-call-making-use-of-user-data-in-ec2/14769/10 "2014-03-24T02:58:53Z")
Ok, so the ec2\_facts module DOES pull down the EC2 information - thanks for that. When keys are created, all hypens (-) are converted to underscores (\_), so the key is ansible\_ec2\_user\_data The value of this key still comes down as “a string, which happens to be json”, so it doesn’t work to ask for {{ ansible\_ec2\_user\_data.hostname }} However, adding in the “from\_json” filter to that does work, and got me to this final solution: - action: ec2\_facts - hostname: name={{ (ansible\_ec2\_user\_data|from\_json).hostname }} Thanks for all the tips - putting them together, and going through the source code, got me to this final solution ! Ben