# Custom Module Output parsing

**URL:** <https://forum.ansible.com/t/custom-module-output-parsing/25196>\
**Category:** Ansible Project\
**Created:** [August 7, 2017, 5:14pm UTC](https://forum.ansible.com/t/custom-module-output-parsing/25196 "2017-08-07T17:14:55Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Adam](https://avatars.discourse-cdn.com/v4/letter/a/4af34b/32.png) [@Adam](https://forum.ansible.com/u/Adam)\
**Post date:** [August 7, 2017, 5:14pm UTC](https://forum.ansible.com/t/custom-module-output-parsing/25196/1 "2017-08-07T17:14:55Z")

</div>

Hi all -

I’m working on a custom module in python that takes a subset of an XML config, and turns it into JSON so I can use it with an Ansible playbook & Jinja2 template. I want to make sure my output will be compatible with Ansible playbook processing. Here’s the output:

[  
{“q1”: {“flowcontrol-start-queuesize”: “-1”, “propertyCount”: 5, “cleanup-interval”: “-1”, “cache-size”: “5000”, “persistence-mode”: “non\_persistent”}},  
{“q2”: {“flowcontrol-start-queuesize”: “-1”, “propertyCount”: 5, “cleanup-interval”: “-1”, “cache-size”: “5000”, “persistence-mode”: “non\_persistent”}},  
{“test”: {“propertyCount”: 1}}  
]

I’ve added line-breaks to make it more readable. Can this output be used in an Ansible playbook? Possibly with the with\_items loop?

Thanks,  
Adam

---

<div class="post-metadata">

**Author:** ![Adam](https://avatars.discourse-cdn.com/v4/letter/a/4af34b/32.png) [@Adam](https://forum.ansible.com/u/Adam)\
**Post date:** [August 7, 2017, 6:57pm UTC](https://forum.ansible.com/t/custom-module-output-parsing/25196/2 "2017-08-07T18:57:18Z")

</div>

My python code works fine to parse the output, but my module continually fails for reasons I can’t explain. Can someone give me a hint?

**Module code:**

#!/usr/bin/python

ANSIBLE\_METADATA = {  
‘metadata\_version’: ‘1.0’,  
‘status’: [‘preview’],  
‘supported\_by’: ‘curated’  
}

import os  
import shutil  
import tempfile  
import traceback  
import xml.etree.cElementTree as etree  
import json

from ansible.module\_utils.basic import AnsibleModule  
from ansible.module\_utils.\_text import to\_bytes, to\_native

from collections import defaultdict

class Xml2Dict(dict):  
def **init** (self, parent\_element):  
if parent\_element.items():  
self.updateDict( dict(parent\_element.items()) )  
for element in parent\_element:  
if len(element):  
aDict = Xml2Dict(element)  
self.updateDict({element.tag: aDict})  
elif element.items(): # items() is special for attributes  
elementattrib= element.items()  
if element.text:  
elementattrib.append((element.tag,element.text )) # add tag:text if exist  
self.updateDict({element.tag: dict(elementattrib)})  
else:  
self.updateDict({element.tag: element.text})

def updateDict (self, aDict ):  
for key in aDict.keys(): # keys() includes tag and attributes  
if key in self:  
value = self.pop(key)  
if type(value) is not list:  
listOfDicts =   
listOfDicts.append(value)  
listOfDicts.append(aDict[key])  
self.update({key: listOfDicts})  
else:  
value.append(aDict[key])  
self.update({key: value})  
else:  
self.update({key:aDict[key]}) # it was self.update(aDict)

def run\_module():

# define the available arguments/parameters that a user can pass to

# the module

module\_args = dict(  
src=dict(type=‘path’)  
)

# seed the result dict in the object

# we primarily care about changed and state

# change is if this module effectively modified the target

# state will include any data that you want your module to pass back

# for consumption, for example, in a subsequent task

result = dict(  
changed=False,  
original\_message=‘’,  
message=‘’,  
failed=‘’  
)

# the AnsibleModule object will be our abstraction working with Ansible

# this includes instantiation, a couple of common attr would be the

# args/params passed to the execution, as well as if the module

# supports check mode

module = AnsibleModule(  
argument\_spec=module\_args,  
supports\_check\_mode=True  
)

src = module.params[‘src’]

# b\_src = to\_bytes(src, errors=‘surrogate\_or\_strict’)

# if not os.path.exists(b\_src):

# module.fail\_json(msg=“Source %s not found” % (src))

# if not os.access(b\_src, os.R\_OK):

# module.fail\_json(msg=“Source %s not readable” % (src))

# if os.path.isdir(b\_src):

# module.fail\_json(msg=“Directory specified as the source instead of a file: %s” % (src))

# if os.path.exists(b\_src):

# b\_src = os.path.realpath(b\_src)

# src = to\_native(b\_src, errors=‘surrogate\_or\_strict’)

# if os.access(b\_src, os.R\_OK):

tree = etree.ElementTree(file=src)  
#print tree.getroot()  
root = tree.getroot()  
#print “tag=%s, attrib=%s” % (root.tag, root.attrib)

from pprint import pprint  
#router = pprint(etree\_to\_dict(root))  
router = Xml2Dict(root)

for key in router[‘swiftlet’][4].iteritems():  
if key[0] == ‘aliases’:  
for item in enumerate(key[1][‘alias’]):  
dest = str(item[1][‘map-to’])  
name = str(item[1][‘name’])

# print dest + “:” + name

queuesList =   
for key in router[‘swiftlet’][9].iteritems():  
if key[0] == ‘queues’:  
for item in enumerate(key[1][‘queue’]):  
name = str(item[1][‘name’])  
attrList = item[1]  
print attrList  
queueDict = {}

if len(attrList) == 1:  
queueDict[name] = {“propertyCount” : len(attrList)}  
else:  
for attrib, value in attrList.iteritems():  
if attrib != “name”:  
if name in queueDict:  
queueDict[name][attrib] = value  
else:  
queueDict[name] = {attrib : value}  
if name in queueDict:  
queueDict[name][“propertyCount”] = len(attrList)  
else:  
queueDict[name] = {“propertyCount” : len(attrList)}

queuesList.append(queueDict)

output = json.dumps(queuesList)

# if the user is working with this module in only check mode we do not

# want to make any changes to the environment, just return the current

# state with no modifications

# if module.check\_mode:

# return result

# manipulate or modify the state as needed (this is going to be the

# part where your module will do what it needs to do)

result[‘original\_message’] = module.params[‘src’]  
result[‘message’] = ‘goodbye’

# use whatever logic you need to determine whether or not this module

# made any modifications to your target

# if module.params[‘src’]:

# result[‘changed’] = True

# during the execution of the module, if there is an exception or a

# conditional state that effectively causes a failure, run

# AnsibleModule.fail\_json() to pass in the message and the result

# if module.params[‘src’] == ‘fail me’:

# module.fail\_json(msg=‘You requested this to fail’, \*\*result)

# in the event of a successful module execution, you will want to

# simple AnsibleModule.exit\_json(), passing the key/value results

module.exit\_json(  
changed=True,  
failed=False)

def main():  
run\_module()

if **name** == ‘ **main** ’:  
main()

**Results:**

TASK [test module] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
task path: /home/ec2-user/ansible/testmod.yml:7  
Using module\_utils file /home/ec2-user/ansible/lib/ansible/module\_utils/\_text.py  
Using module\_utils file /home/ec2-user/ansible/lib/ansible/module\_utils/basic.py  
Using module\_utils file /home/ec2-user/ansible/lib/ansible/module\_utils/six/ **init**.py  
Using module\_utils file /home/ec2-user/ansible/lib/ansible/module\_utils/parsing/convert\_bool.py  
Using module\_utils file /home/ec2-user/ansible/lib/ansible/module\_utils/parsing/ **init**.py  
Using module\_utils file /home/ec2-user/ansible/lib/ansible/module\_utils/pycompat24.py  
Using module file /home/ec2-user/ansible/lib/ansible/modules/messaging/new\_module.py  
\<127.0.0.1\> ESTABLISH LOCAL CONNECTION FOR USER: ec2-user  
\<127.0.0.1\> EXEC /bin/sh -c ‘echo ~ && sleep 0’  
\<127.0.0.1\> EXEC /bin/sh -c ‘( umask 77 && mkdir -p “`echo /home/ec2-user/.ansible/tmp/ansible-tmp-1502132102.2-141765875120224`” && echo ansible-tmp-1502132102.2-141765875120224=“`echo /home/ec2-user/.ansible/tmp/ansible-tmp-1502132102.2-141765875120224`” ) && sleep 0’  
\<127.0.0.1\> PUT /tmp/tmpHHjMUg TO /home/ec2-user/.ansible/tmp/ansible-tmp-1502132102.2-141765875120224/new\_module.py  
\<127.0.0.1\> EXEC /bin/sh -c ‘chmod u+x /home/ec2-user/.ansible/tmp/ansible-tmp-1502132102.2-141765875120224/ /home/ec2-user/.ansible/tmp/ansible-tmp-1502132102.2-141765875120224/new\_module.py && sleep 0’  
\<127.0.0.1\> EXEC /bin/sh -c ‘/home/ec2-user/ansible/venv/bin/python /home/ec2-user/.ansible/tmp/ansible-tmp-1502132102.2-141765875120224/new\_module.py; rm -rf “/home/ec2-user/.ansible/tmp/ansible-tmp-1502132102.2-141765875120224/” \> /dev/null 2\>&1 && sleep 0’  
fatal: [localhost]: FAILED! =\> {  
“changed”: false,  
“failed”: true,  
“module\_stderr”: “”,  
“module\_stdout”: “{‘cache-size’: ‘5000’, ‘flowcontrol-start-queuesize’: ‘-1’, ‘name’: ‘q1’, ‘cleanup-interval’: ‘-1’, ‘persistence-mode’: ‘non\_persistent’}\n{‘cache-size’: ‘5000’, ‘flowcontrol-start-queuesize’: ‘-1’, ‘name’: ‘q2’, ‘cleanup-interval’: ‘-1’, ‘persistence-mode’: ‘non\_persistent’}\n{‘cache-size’: ‘5000’, ‘flowcontrol-start-queuesize’: ‘-1’, ‘name’: ‘q3’, ‘cleanup-interval’: ‘-1’, ‘persistence-mode’: ‘non\_persistent’}\n{‘name’: ‘test’}\n\n{"invocation": {"module\_args": {"src": "/home/ec2-user/ansible/routerconfig.xml"}}, "failed": false, "changed": true}\n”,  
“msg”: “MODULE FAILURE”,  
“rc”: 0  
}  
to retry, use: --limit @/home/ec2-user/ansible/testmod.retry

PLAY RECAP \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
localhost : ok=1 changed=0 unreachable=0 failed=1

---

<div class="post-metadata">

**Author:** ![Adam](https://avatars.discourse-cdn.com/v4/letter/a/4af34b/32.png) [@Adam](https://forum.ansible.com/u/Adam)\
**Post date:** [August 7, 2017, 7:00pm UTC](https://forum.ansible.com/t/custom-module-output-parsing/25196/3 "2017-08-07T19:00:23Z")

</div>

My playbook is below:

- name: test  
gather\_facts: yes  
hosts: localhost  
connection: local

tasks:

- name: test module  
new\_module:  
src: /home/ec2-user/ansible/routerconfig.xml  
register: testout

- name: debug  
debug:  
msg: ‘{{testout}}’

---

<div class="post-metadata">

**Author:** ![Kai\_Stian\_Olstad](https://avatars.discourse-cdn.com/v4/letter/k/e19adc/32.png) [@Kai\_Stian\_Olstad](https://forum.ansible.com/u/Kai_Stian_Olstad)\
**Post date:** [August 8, 2017, 1:00pm UTC](https://forum.ansible.com/t/custom-module-output-parsing/25196/4 "2017-08-08T13:00:55Z")

</div>

For development question you might have better luck in the development mailing list [https://groups.google.com/forum/#!forum/ansible-devel](https://groups.google.com/forum/#!forum/ansible-devel)

---

<div class="post-metadata">

**Author:** ![Adam](https://avatars.discourse-cdn.com/v4/letter/a/4af34b/32.png) [@Adam](https://forum.ansible.com/u/Adam)\
**Post date:** [August 8, 2017, 5:26pm UTC](https://forum.ansible.com/t/custom-module-output-parsing/25196/5 "2017-08-08T17:26:28Z")

</div>

thanks