Playing with Ansible mutiple callbacks

Tom_Laredo · January 11, 2013, 4:44pm

Hi,

As I began to introduce my projet in another post, I’m currently experimenting a blocking issue.
The thing that I want to do is pretty simple and I think that I am verry close to the solution but I clearly need your help

Here is the flow I want to create :
A playbook executes a module (that I quickly wrote) which runs the sell command apt-show-versions, parse the result and then return it into JSON format.
At the end, a callback is responsible for processing the result and then puting it into a database.

Info: this (great !) web page helped me to do it : http://jpmens.net/2012/09/11/watching-ansible-at-work-callbacks/
It almost works but I’m still have an issue (see under).

So here are the files:

ANSIBLE PLAYBOOK:

hosts: webservers
user: root
vars:
name: dpkg Tom
tasks:

name: Verify that apt-show-versions is installed and install it if it is not present
action: apt pkg=apt-show-versions state=present

name: Launch a custom dpkg-show-versions module
action: dpkg_tom

ANSIBLE MODULE (in bash language because I don’t know how to do it in Python):

DEFINITION DE LA FONCTION PRINCIPALE

main()
{

parseArguments $1

echo -n “{"changed": "False", "name": "dpkg_tom", "packages": [”

nombre_lignes=$(apt-show-versions --upgradeable | wc -l)
compteur=1

apt-show-versions --upgradeable | while read line
do
echo “$line” | sed ‘s@^(.?)/(.?) (.?) from (.?) to (.*)@{"\1":{"distribution": "\2","status": "\3","installed_version": "\4","available_version": "\5"@’;

echo -ne “}}”

if [ $compteur -lt $nombre_lignes ]
then
echo -ne “,”
fi

compteur=$[$compteur + 1]
done

echo -ne “"":{”;

echo -n “]}”
}

######################################################

main

Info : this module works just fine (tested without the callback).

ANSIBLE CALLBACK:

import os
import time
import sqlite3

dbname = ‘/etc/ansible/cmdb.db’
TIME_FORMAT=‘%Y-%m-%d %H:%M:%S’

try:
con = sqlite3.connect(dbname)
cur = con.cursor()
except:
pass

def log(host, data):

if type(data) == dict:
invocation = data.pop(‘invocation’, None)
if invocation.get(‘module_name’, None) != ‘setup’:
return

facts = data.get(‘ansible_facts’, None)

now = time.strftime(TIME_FORMAT, time.localtime())

try:

host is a unique index

cur.execute(“REPLACE INTO inventory (now, host, arch, dist, distvers, sys,kernel) VALUES(?,?,?,?,?,?,?);”,
(
now,
facts.get(‘ansible_hostname’, None),
facts.get(‘ansible_architecture’, None),
facts.get(‘ansible_distribution’, None),
facts.get(‘ansible_distribution_version’, None),
facts.get(‘ansible_system’, None),
facts.get(‘ansible_kernel’, None),
))
con.commit()
except:
pass

class CallbackModule(object):
def runner_on_ok(self, host, res):
log(host, res)

The problem that I have is the following Python error (error raised during callback execution):

fatal: [127.0.0.1] => Traceback (most recent call last):
File “/usr/lib/pymodules/python2.6/ansible/runner/init.py”, line 236, in _executor
exec_rc = self._executor_internal(host)
File “/usr/lib/pymodules/python2.6/ansible/runner/init.py”, line 292, in _executor_internal
return self._executor_internal_inner(host, self.module_name, self.module_args, inject, port)
File “/usr/lib/pymodules/python2.6/ansible/runner/init.py”, line 435, in _executor_internal_inner
self.callbacks.on_ok(host, data)
File “/usr/lib/pymodules/python2.6/ansible/callbacks.py”, line 363, in on_ok
super(PlaybookRunnerCallbacks, self).on_ok(host, host_result)
File “/usr/lib/pymodules/python2.6/ansible/callbacks.py”, line 191, in on_ok
call_callback_module(‘runner_on_ok’, host, res)
File “/usr/lib/pymodules/python2.6/ansible/callbacks.py”, line 50, in call_callback_module
method(*args, **kwargs)
File “/usr/lib/pymodules/python2.6/ansible/callback_plugins/inventory.py”, line 43, in runner_on_ok
log(host, res)
File “/usr/lib/pymodules/python2.6/ansible/callback_plugins/inventory.py”, line 18, in log
if invocation.get(‘module_name’, None) != ‘setup’:
AttributeError: ‘NoneType’ object has no attribute ‘get’

FATAL: all hosts have already failed – aborting

Then, I think that this issue is raised because of the lack of module_name in the result that my module give.
Have I to declare my custom module somewhere to make ansible know the module_name of my module?
Have I to put a module_name attribute in my JSON result?
I know that I have a verry poor level in Python and that I’m a beginner in playing with Ansible features so, here I am

Thank you for you awsome project anyway!
Tom

Michael_DeHaan · January 11, 2013, 5:05pm

Nope, it's saying there is no 'invocation' hash at all in what you
have run. This is probably a minor quirk that should not be the
case, but see example here:

https://github.com/ansible/ansible/blob/devel/plugins/callbacks/log_plays.py

I suspect adding the if clause about 'verbose_override' will fix you up.

Tom_Laredo · January 11, 2013, 5:18pm

Thanks for your verry impressive reactivity which is really appreciated!
I see the condition that you’re talking about, but I’m not sure to understand.
Could you please tell me what you mean by “invocation hash”?
Have I to declare something in my module?
The “setup” module (wrote in Python) don’t causes any problem with the callback I gave below so how can I say "this callback has to work ONLY IF the name is ‘setup’ " if my module don’t have a proper name?

Thank you for your help
Tom

Dusan_Sovic · January 11, 2013, 5:37pm

Hi Tom,

try something like this in your callback code (just put if invocation is None: and return…):

if type(data) == dict:
invocation = data.pop(‘invocation’, None)
if invocation is None:
return
if invocation.get(‘module_name’, None) != ‘setup’:
return

hope it helps you,
Dusan

Dňa piatok, 11. januára 2013 18:18:28 UTC+1 grobs napísal(-a):

Tom_Laredo · January 14, 2013, 10:06am

Thank you verry much! It works properly

Now, how can I do if I want to write another callback which is executed when my module (written in bash) return it’s result?
Because if my module dos’nt have any name, I don’t see how I can link it to a callback in particular.

Tom

Tom_Laredo · January 15, 2013, 4:58pm

For information, I just rewritten the module in perl to make it just a bit better.
Here is the source code:
8<---------------------------------------------------------------------------------------------------------------------------

#!/usr/bin/perl

use strict;
use warnings;

sub main
{
print “{"changed": "False", "packages": [”;

my $nbrLignes=apt-show-versions | wc -l;
my $compteur=0;

for(apt-show-versions) {
$compteur++;

Formats attendus :

zlib1g/squeeze uptodate 1:1.2.3.4.dfsg-3

libnss3-1d/squeeze upgradeable from 3.12.8-1+squeeze5 to 3.12.8-1+squeeze6

ansible-provisioning 0.9 installed: No available version in archive

next if not $_ =~ /
^
([a-z0-9_-]?) # package
/ # slash
(.?) # distribution
\s #
(.?) # status
\s #
(?: # groupe non capturant
from # from
\s #
(.?) # installed_version
\s #
to # to
\s #
)? # fin de groupe non capturant
(.+$) # available_version
/ix; # case Insensitive and eXtended spacing

my $package = $1;
my $distribution = $2;
my $status = $3;
my $installed_version = ‘’;
my $available_version = ‘’;

if(defined $4) {
$installed_version = $4;
$available_version = $5;
}
else {
$installed_version = $5;
}

print “{"$package":{"distribution": "$distribution","status": "$status","installed_version": "$installed_version","available_version": "$available_version"}}”;

if($compteur < $nbrLignes) {
print “,”
}
}

print “]}”;
}

######################################################

main;

8<---------------------------------------------------------------------------------------------------------------------------

This module works fine in my playbook but I don’t understand how I could give it a name to identify the module in my callback and then use the result of my module.

Any idea?

Tom

Michael_DeHaan · January 15, 2013, 5:21pm

Callback modules must be written in Python, so I'm fairly confused here.

Regular ansible modules which are executed on remote hosts are
completely different from callbacks.

Tom_Laredo · January 15, 2013, 5:40pm

Yes, I’m ok with that I think, sorry if I’m not clear in what I say.

The source code that I gave before is the module, not the callback.
The module read the result of apt-show-versions and then return JSON formatted result.
I also have a callback (which is written in Python) which take the result and then put it in a database.
As I have several modules and several callback associated, I want to make my Python callback work ony if this or this module was launched.
To do this, I have those lines in my Python Callback to identify the module which was ran:
8<---------------------------------------------------------------------------------------------------------------------------

if type(data) == dict:
invocation = data.pop(‘invocation’, None)
if invocation is None:
return
if invocation.get(‘module_name’, None) != ‘the_name_of_my_module’:
return

8<---------------------------------------------------------------------------------------------------------------------------

But as my module is written in Perl (or bash or whatever is not Python), I don’t know how to give a name to my module to verify in my callback that the modulethat was ran really is the one I want (“the_name_of_my_module” here).

Hope that my question is more understandable.

Tom

Michael_DeHaan · January 15, 2013, 6:14pm

Yes, I'm ok with that I think, sorry if I'm not clear in what I say.

The source code that I gave before is the module, not the callback.
The module read the result of apt-show-versions and then return JSON
formatted result.
I also have a callback (which is written in Python) which take the result
and then put it in a database.
As I have several modules and several callback associated, I want to make my
Python callback work ony if this or this module was launched.
To do this, I have those lines in my Python Callback to identify the module
which was ran:
8<---------------------------------------------------------------------------------------------------------------------------

if type(data) == dict:
        invocation = data.pop('invocation', None)
        if invocation is None:
            return
        if invocation.get('module_name', None) != 'the_name_of_my_module':
            return

8<---------------------------------------------------------------------------------------------------------------------------

But as my module is written in Perl (or bash or whatever is not Python), I
don't know how to give a name to my module to verify in my callback that the
modulethat was ran really is the one I want ("the_name_of_my_module" here).

Hope that my question is more understandable.

I see.

Looks like if your module is named 'foo', it doesn't matter what
language it is, it will still appear as foo.

line 423 here

https://github.com/ansible/ansible/blob/devel/lib/ansible/runner/__init__.py

Topic		Views
Get data to MySQL database with callback Ansible Project	5	February 28, 2013
Getting two results with custom callbacks Ansible Project	2	April 14, 2018
Call back script problems Ansible Project	2	March 5, 2016
More informations logs Ansible Project	4	February 20, 2013
Ansible API Ansible Project	3	April 19, 2013

Playing with Ansible mutiple callbacks

DEFINITION DE LA FONCTION PRINCIPALE

parseArguments $1

echo -ne “"":{”;

host is a unique index

Formats attendus :

zlib1g/squeeze uptodate 1:1.2.3.4.dfsg-3

libnss3-1d/squeeze upgradeable from 3.12.8-1+squeeze5 to 3.12.8-1+squeeze6

ansible-provisioning 0.9 installed: No available version in archive

Related topics

`host` is a unique index