Custom Module Output parsing

Hi all -

I’m working on a custom module in python that takes a subset of an XML config, and turns it into JSON so I can use it with an Ansible playbook & Jinja2 template. I want to make sure my output will be compatible with Ansible playbook processing. Here’s the output:

[
{“q1”: {“flowcontrol-start-queuesize”: “-1”, “propertyCount”: 5, “cleanup-interval”: “-1”, “cache-size”: “5000”, “persistence-mode”: “non_persistent”}},
{“q2”: {“flowcontrol-start-queuesize”: “-1”, “propertyCount”: 5, “cleanup-interval”: “-1”, “cache-size”: “5000”, “persistence-mode”: “non_persistent”}},
{“test”: {“propertyCount”: 1}}
]

I’ve added line-breaks to make it more readable. Can this output be used in an Ansible playbook? Possibly with the with_items loop?

Thanks,
Adam

My python code works fine to parse the output, but my module continually fails for reasons I can’t explain. Can someone give me a hint?

Module code:

#!/usr/bin/python

ANSIBLE_METADATA = {
‘metadata_version’: ‘1.0’,
‘status’: [‘preview’],
‘supported_by’: ‘curated’
}

import os
import shutil
import tempfile
import traceback
import xml.etree.cElementTree as etree
import json

from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils._text import to_bytes, to_native

from collections import defaultdict

class Xml2Dict(dict):
def init(self, parent_element):
if parent_element.items():
self.updateDict( dict(parent_element.items()) )
for element in parent_element:
if len(element):
aDict = Xml2Dict(element)
self.updateDict({element.tag: aDict})
elif element.items(): # items() is special for attributes
elementattrib= element.items()
if element.text:
elementattrib.append((element.tag,element.text )) # add tag:text if exist
self.updateDict({element.tag: dict(elementattrib)})
else:
self.updateDict({element.tag: element.text})

def updateDict (self, aDict ):
for key in aDict.keys(): # keys() includes tag and attributes
if key in self:
value = self.pop(key)
if type(value) is not list:
listOfDicts =
listOfDicts.append(value)
listOfDicts.append(aDict[key])
self.update({key: listOfDicts})
else:
value.append(aDict[key])
self.update({key: value})
else:
self.update({key:aDict[key]}) # it was self.update(aDict)

def run_module():

define the available arguments/parameters that a user can pass to

the module

module_args = dict(
src=dict(type=‘path’)
)

seed the result dict in the object

we primarily care about changed and state

change is if this module effectively modified the target

state will include any data that you want your module to pass back

for consumption, for example, in a subsequent task

result = dict(
changed=False,
original_message=‘’,
message=‘’,
failed=‘’
)

the AnsibleModule object will be our abstraction working with Ansible

this includes instantiation, a couple of common attr would be the

args/params passed to the execution, as well as if the module

supports check mode

module = AnsibleModule(
argument_spec=module_args,
supports_check_mode=True
)

src = module.params[‘src’]

b_src = to_bytes(src, errors=‘surrogate_or_strict’)

if not os.path.exists(b_src):

module.fail_json(msg=“Source %s not found” % (src))

if not os.access(b_src, os.R_OK):

module.fail_json(msg=“Source %s not readable” % (src))

if os.path.isdir(b_src):

module.fail_json(msg=“Directory specified as the source instead of a file: %s” % (src))

if os.path.exists(b_src):

b_src = os.path.realpath(b_src)

src = to_native(b_src, errors=‘surrogate_or_strict’)

if os.access(b_src, os.R_OK):

tree = etree.ElementTree(file=src)
#print tree.getroot()
root = tree.getroot()
#print “tag=%s, attrib=%s” % (root.tag, root.attrib)

from pprint import pprint
#router = pprint(etree_to_dict(root))
router = Xml2Dict(root)

for key in router[‘swiftlet’][4].iteritems():
if key[0] == ‘aliases’:
for item in enumerate(key[1][‘alias’]):
dest = str(item[1][‘map-to’])
name = str(item[1][‘name’])

print dest + “:” + name

queuesList =
for key in router[‘swiftlet’][9].iteritems():
if key[0] == ‘queues’:
for item in enumerate(key[1][‘queue’]):
name = str(item[1][‘name’])
attrList = item[1]
print attrList
queueDict = {}

if len(attrList) == 1:
queueDict[name] = {“propertyCount” : len(attrList)}
else:
for attrib, value in attrList.iteritems():
if attrib != “name”:
if name in queueDict:
queueDict[name][attrib] = value
else:
queueDict[name] = {attrib : value}
if name in queueDict:
queueDict[name][“propertyCount”] = len(attrList)
else:
queueDict[name] = {“propertyCount” : len(attrList)}

queuesList.append(queueDict)

output = json.dumps(queuesList)

if the user is working with this module in only check mode we do not

want to make any changes to the environment, just return the current

state with no modifications

if module.check_mode:

return result

manipulate or modify the state as needed (this is going to be the

part where your module will do what it needs to do)

result[‘original_message’] = module.params[‘src’]
result[‘message’] = ‘goodbye’

use whatever logic you need to determine whether or not this module

made any modifications to your target

if module.params[‘src’]:

result[‘changed’] = True

during the execution of the module, if there is an exception or a

conditional state that effectively causes a failure, run

AnsibleModule.fail_json() to pass in the message and the result

if module.params[‘src’] == ‘fail me’:

module.fail_json(msg=‘You requested this to fail’, **result)

in the event of a successful module execution, you will want to

simple AnsibleModule.exit_json(), passing the key/value results

module.exit_json(
changed=True,
failed=False)

def main():
run_module()

if name == ‘main’:
main()

Results:

TASK [test module] *******************************************************************************************************************************************
task path: /home/ec2-user/ansible/testmod.yml:7
Using module_utils file /home/ec2-user/ansible/lib/ansible/module_utils/_text.py
Using module_utils file /home/ec2-user/ansible/lib/ansible/module_utils/basic.py
Using module_utils file /home/ec2-user/ansible/lib/ansible/module_utils/six/init.py
Using module_utils file /home/ec2-user/ansible/lib/ansible/module_utils/parsing/convert_bool.py
Using module_utils file /home/ec2-user/ansible/lib/ansible/module_utils/parsing/init.py
Using module_utils file /home/ec2-user/ansible/lib/ansible/module_utils/pycompat24.py
Using module file /home/ec2-user/ansible/lib/ansible/modules/messaging/new_module.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: ec2-user
<127.0.0.1> EXEC /bin/sh -c ‘echo ~ && sleep 0’
<127.0.0.1> EXEC /bin/sh -c ‘( umask 77 && mkdir -p “echo /home/ec2-user/.ansible/tmp/ansible-tmp-1502132102.2-141765875120224” && echo ansible-tmp-1502132102.2-141765875120224=“echo /home/ec2-user/.ansible/tmp/ansible-tmp-1502132102.2-141765875120224” ) && sleep 0’
<127.0.0.1> PUT /tmp/tmpHHjMUg TO /home/ec2-user/.ansible/tmp/ansible-tmp-1502132102.2-141765875120224/new_module.py
<127.0.0.1> EXEC /bin/sh -c ‘chmod u+x /home/ec2-user/.ansible/tmp/ansible-tmp-1502132102.2-141765875120224/ /home/ec2-user/.ansible/tmp/ansible-tmp-1502132102.2-141765875120224/new_module.py && sleep 0’
<127.0.0.1> EXEC /bin/sh -c ‘/home/ec2-user/ansible/venv/bin/python /home/ec2-user/.ansible/tmp/ansible-tmp-1502132102.2-141765875120224/new_module.py; rm -rf “/home/ec2-user/.ansible/tmp/ansible-tmp-1502132102.2-141765875120224/” > /dev/null 2>&1 && sleep 0’
fatal: [localhost]: FAILED! => {
“changed”: false,
“failed”: true,
“module_stderr”: “”,
“module_stdout”: “{‘cache-size’: ‘5000’, ‘flowcontrol-start-queuesize’: ‘-1’, ‘name’: ‘q1’, ‘cleanup-interval’: ‘-1’, ‘persistence-mode’: ‘non_persistent’}\n{‘cache-size’: ‘5000’, ‘flowcontrol-start-queuesize’: ‘-1’, ‘name’: ‘q2’, ‘cleanup-interval’: ‘-1’, ‘persistence-mode’: ‘non_persistent’}\n{‘cache-size’: ‘5000’, ‘flowcontrol-start-queuesize’: ‘-1’, ‘name’: ‘q3’, ‘cleanup-interval’: ‘-1’, ‘persistence-mode’: ‘non_persistent’}\n{‘name’: ‘test’}\n\n{"invocation": {"module_args": {"src": "/home/ec2-user/ansible/routerconfig.xml"}}, "failed": false, "changed": true}\n”,
“msg”: “MODULE FAILURE”,
“rc”: 0
}
to retry, use: --limit @/home/ec2-user/ansible/testmod.retry

PLAY RECAP ***************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=1

My playbook is below:

  • name: test
    gather_facts: yes
    hosts: localhost
    connection: local

tasks:

  • name: test module
    new_module:
    src: /home/ec2-user/ansible/routerconfig.xml
    register: testout

  • name: debug
    debug:
    msg: ‘{{testout}}’

For development question you might have better luck in the development mailing list https://groups.google.com/forum/#!forum/ansible-devel

thanks