Kerberos domain authentication for Windows hosts

Archives Ansible Developer
,
1

Hi,

I’ve been looking at adding support for Kerberos for deployments to Windows hosts in Ansible/Ansible Tower.

There are 2 open PR on the subject:

The first one suggested that we support kerberos authentication by changing the ansible_ssh_user to “ansible_kerberos”. Ansible would then use the current default kerberos principal set by a previous call to kinit.
As pointed in the PR discussion, there is actually a need for non-default kerberos credentials; ie you might want to run a playbook with some tasks for server A as user alice@ad.domain.com and server B as bob@as.domain.com

To cater for that ianclegg suggested we use the following convention:

Kerberos single sign on
ansible_ssh_user: <not specified>
ansible_ssh_pass: <not specified>

K****erberos with a specific user (UPN)

ansible_ssh_user: user@realm
ansible_ssh_pass:

Kerberos with a specific username password
ansible_ssh_user: user@realm
ansible_ssh_pass: password

I think we should support all 3 cases, and this suggestion seems reasonable.
Unfortunately this PR went a bit quiet, and nothing happened in the last few weeks. In particular no suggestion was made as to how to implement this.

#8914 was then submitted and suggested to have Ansible itself call kinit to ensure the provided credentials could be used in the Kerberos authentication.

One thing I really like about it is that it also adds an “ansible_authorization_type” variable, which explicitly specifies the required authentication mechanism, which I think is clearer than simply rely on the username to decide whether to use Kerberos. As an added bonus it would allow 2 other ways to indicate a user on the default domain:

Kerberos with a specific user on the default domain

ansible_ssh_user: user
ansible_ssh_pass:
ansible_authorization_type: kerberos

Kerberos with a specific username/password on the default domain
ansible_ssh_user: user
ansible_ssh_pass: password
ansible_authorization_type: kerberos

Regarding the call to kinit, I took a different approach and updated pywinrm to support username and password (which are currently ignored if you’re using Kerberos with it).

My ansible repository is https://github.com/nicodeslandes/ansible, and the changes to pywinrm are available there: https://github.com/nicodeslandes/pywinrm/commits/alt_credentials

These changes unfortunately also required some changes to the underlying pykerberos library, so that it would accept a password. That change is here: https://github.com/nicodeslandes/pykerberos/commits/kerberos_passwords_spike.

With these changes, there’s no need to call kinit. pywinrm/pykerberos take care of requesting a TGT for the username/password provided.
This was necessary for me for 2 reasons:

  • In the project I’m working on we plan to use Ansible Tower to automate Ansible deployment. So we cannot call kinit ahead of a deployement
  • For some reason the kerberos usernames in my client’s production environment start with a hyphen; so kinit doesnt’ work there, as it takes the username for an option (!).

I also added an option to allow the delegation of the user’s credentials on the remote server. Useful if for instance you need to access a network share from the Windows server you’re accessing.

Now both the proposed solution in #8914 and the one I implemented in pywinrm/pykerberos have a big security problem in my scenario (using Tower):

If user A provides a password for a Kerberos principal, the TGT for this principal gets added to the credential cache of whichever user executes ansible (‘awx’ in that case).

That means that user B may then start a deployment for the same kerberos principal without having to provide a password, and still have access to a TGT for it from the credential cache.
I haven’t found a solution to this issue yet, but we would need a sort of transient credential cache that would only be used for a specific Ansible playbook run.

I should also mention a comment from AdmiralNemo in the GitHub discussion for #8914, that is very relevant to this discussion:

Also, pywinrm may end up having native support for requesting the tgt given a principal and password[1], so having Ansible call kinit directly would not be necessary in that case.

[1] diyan/pywinrm#6 (comment)

What do people think?

Does this seem like a good way to add support for domain authentication in Ansible?

Does anyone have any idea how to tackle the security issue with cached credentials?

Thanks,

Nico

2

Hi Nicolas,

Long pile of data so not exactly sure how to respond to specific questions.

We’re currently reviewing both of these.

We’re busy looking at various issues - some of them Windows related - some not. But anyway, stay tuned, not hearing anything in a week or two is ok, it is our goal to have nearly most of the Windows tickets merged for 1.8 release timeframe.

Kerberos/domain auth here is important to us and will be a thing.

We’re probably going to not select to merge changes that are not yet incorporated into their upstream projects, but do appreciate pull requests.

If you can link this discussion into those existing tickets it will also be good to keep track of them there.

–Michael

3

Thanks for this, good summary and its worth getting this right.

I’m still thinking about this one. I suppose we could limit the opportunity for tickets being borrowed by only asking for short lifetime tickets ‘kinit -l 3h user@wherever’. It would have to be configurable but I don’t like this though… what if your play takes a bit longer than usual?

It seems calling kdestroy isn’t really an option in all cases as it gets rid of all of your tickets. I suppose if kdestroy meets your needs there’s nothing to stop you adding it as a task in your plays. Also, it still doesn’t tackle user B logging in while user A is halfway through running a play.

I’ll keep thinking

4

Hi Michael,

Yeah, sorry for the long post :). I was trying to summarize what’s been discussed already in the 2 tickets.
Just to be clear I wasn’t suggesting you guys merge anything just yet, if only because of the security issues I highlighted. I just wanted to discuss the options that we have.
Glad to hear domain authentication is on your radar already.

5

Thanks.
I agree with you, none of these solutions are completely satisfactory.

I just saw something last night that might be what we need. MIT Kerberos support a credential cache type called “MEMORY”. Here’s the what the documentation says about these:

MEMORY caches are for storage of credentials that don’t need to be made available outside of the current process

If we can find a way to create a Memory credential cache and associate it to a playbook run somehow, we may be able to solve the security issue, without having to request TGTs for every requests in the playbook.
I’ll try to put together a quick POC along these lines, and see if that works.

Nico

6

I’ve been trying to modify #8914 to get round the security issue Nicolas Deslandes has identified and also an equally pressing issue which occurs when you start running against multiple hosts. Currently I get occasional failures which appear to be because kinit is writing to the credential cache from > 1 thread, which sometimes corrupts the cache information.

What I think I need to be able to do is call kinit exactly once for each winrm host that is configured for kerberos when ansible_playbook starts and when all plays have terminated, call kdestroy. However I could do with pointing in the right direction as I don’t yet dream in python.

Its becoming clear to me now that ultimately I will need to tackle supporting both the situation described above, where a/ ansible is just using kerberos as a connection mechanism, and also b/ the case where the ansible controller is effectively taking part in the same domain as the windows hosts it is controlling (in which case the kerberos credentials will be acquired during login and there won’t be a need to call kinit or kdestroy).

I’m thinking this will probably be another parameter, perhaps something like the following, where a/ would be ‘acquire’ and b/ would be ‘assume’.
ansible_authorization_mode: acquire
or
ansible_authorization_mode: assume

I welcome suggestions for a better name. Which mode ought to be the default?

Jon

7

Hello,

Just wanted to say I’m still working on this. I’m now thinking that rather than adding ansible_authorization_mode the default behaviour would be to assume a ticket is available, but for cases like mine where ansible needs to call kinit to acquire a ticket, I’m creating a callback plugin. I’ve yet to prove all of this but my plan is to make use of the KEYRING type credential cache to keep the cached credentials private to an individual ansible / ansible-playbook run. Info about KEYRING-type credential cache here:

http://web.mit.edu/kerberos/krb5-devel/doc/basic/ccache_def.html

Jon

8

Hi Jon, thanks for continuing to dig into this one. And Nico, thanks for a
good summary of all the kerberos discussion so far.

I've also been looking at kerberos auth, and in my opinion it's mostly
analogous to ssh key auth in that user of ansible should be responsible for
configuring their environment appropriately:

   - If my private key is present in a default location (ssh) or I already
   have a TGT (kerberos), ansible's underlying connection plugin should use it
   to connect.
   - If my private key is encrypted (ssh) or I need to request a TGT
   (kerberos), I am responsible for running ssh-agent/ssh-add or kinit and
   providing a password before running ansible.
   - If I need to use multiple private keys (ssh) or I need TGTs for
   multiple realms (kerberos), I am also responsible for using
   ssh-agent/ssh-add or kinit to setup those credentials prior to running
   ansible.

Likewise, I've also found a few changes to pywinrm are needed to make it
all work.

For Tower support, we already use ssh-agent to handle encrypted private
keys and can respond to the password prompt from ssh-add. I would expect
that Tower could do something similar to run kinit and respond to the
password prompt prior to running a job, as well as provide some isolation
between job runs to prevent keys being inadvertently shared between jobs.

I don't know where kerberos support is in the Tower roadmap, so a
workaround would be needed until then. A callback plugin that did some
initial setup in playbook_on_start and cleaned up in playbook_on_stats
would be one approach that doesn't change any of your playbooks, but it's
not that easy to access your inventory from that callback if you needed to
reference any host variables. A custom module or action plugin that ran
locally at the beginning of your playbook would be able to call kinit for
each host that needs it or use pykerberos directly to obtain a TGT.

9

Chris,

Thanks for this, sounds like we are largely in agreement.

I’m midway through creating a callback plugin. I’ve been basing it on the #8194 PR so assuming I can find hosts which might need kerberos using ansible_authorization_type: kerberos
set in my inventory. In on_setup I can get hosts like this:

hosts = self.playbook.inventory.get_hosts()

for host in hosts:

host_vars = self.playbook.inventory.get_variables(host.name)
for var in host_vars:
for var in host_vars:

check for TGT etc…

I have some as-yet-untested code to look for TGTs using klist and some more to request a new ticket or refresh an existing one. I need to tackle date parsing so I can detect aged tickets that need renewing (from what I can tell ssh keys don’t seem to have an inherently limited lifespan that kerberos tickets do although its not something I have made a great deal of use of yet).

I’m interested to know whether you are thinking along the lines of having an optional host var like ansible_authorization_type: kerberos
or whether you would be thinking of detecting the need for kerberos in some way - I am happy having custom plugins to support the particular use cases I need but I’m a lot less keen on having to run on a modified version of ansible.

Jon

10

I'm leaning against an extra parameter for authorization type, but need to
to a little more experimenting before I consider that my final answer.

11

Hi, Chris.

Thanks for this. I am not passionate about the extra parameter, having something declared makes it easy to detect when to try using kerberos to connect, but a convention might work (perhaps having an ansible_ssh_user containing and ‘@’ or a '' would be enough?) I also briefly entertained the idea of having a ‘winrm-domain’ connection plugin, but it would be so similar to the existing winrm plugin I went off that idea as soon as I looked at the code.

Anyway, I thought I’d post up a very rough first pass at the plugin I mentioned above in case it is of any use to anyone interested in this.

By ‘very rough’ I mean it hasn’t been tested in any meaningful way, has lots of TODOs still to address and currently still depends on having the extra ansible_authorization_type: kerberos parameter

I am hoping to get some time this week to knock the roughest edges off it before I try and create a PR for it if it actually pans out.

Jon

`
$ cat tickets_plugin.py

This plugin for ansible is intended to manage acquiring and destroying

kerberos or windows domain credentials.

Use at your own risk, not yet subjected to any meaningful testing.

import os
import json
import subprocess
import csv
import datetime
from datetime import datetime, timedelta

from subprocess import Popen, PIPE
from ansible import errors

class CallbackModule(object):
“”"
This callback module is intended to manage acquiring and disposing
of kerberos / windows domain credentials.
“”"

def init(self):
print “Windows/Kerberos Domain-joining plugin is active.”

TODO check kerberos user package has been installed here

and disable the plugin and warn use if so.

def check_if_ticket_is_still_usable(self, tgt_info):
start = tgt_info[‘startDate’] + ’ ’ + tgt_info[‘startTime’]
expires = tgt_info[‘expiresDate’] + ’ ’ + tgt_info[‘expiresTime’]
ticket_valid_from = datetime.strptime(start, ‘%d/%m/%y %H:%M:%S’)
ticket_valid_to = datetime.strptime(expires, ‘%d/%m/%y %H:%M:%S’)
now = datetime.now()
ticket_aged_after= now - timedelta(days=1)
remaining_ticket_life = ticket_valid_to - now
if now > ticket_valid_to:
print “Ticket Expired”
if now < ticket_valid_to and now > ticket_valid_from:
print “Ticket valid for %s days, %s hours, %s minutes” % (days_hours_minutes(remaining_ticket_life))
if now > ticket_aged_after:
print “Ticket is aged”

def check_if_ticket_is_for_this_domain(self, tgt_info, userAtDomain):
principle = tgt_info[‘servicePrincipal’]
parts = userAtDomain.split(‘@’, 2)
domain = parts[1].upper()
if ( principle.startswith(‘krbtgt’) and principle.endswith(domain) ):
return True
return False

def find_or_acquire_ticket(self, userAtDomain, password):
“”"

look for a ticket for configured ansible_ssh_user / ansible_ssh_password

todo open subprocesses (klist, kinit, kdestroy) the same way.

“”"
process = subprocess.Popen([‘klist’], stdout=subprocess.PIPE)
stdout, stderr = process.communicate()
stdout_lines = stdout.decode(‘ascii’).splitlines()
if not stdout_lines:
print "No credentials cache found. Attempting to get a ticket to cache… "
self.cache_new_ticket(userAtDomain, password)
return
else:
ticket_info_line =
ticket_info_line.append(all_lines[4]) # all the interesting info is on line 4
reader = csv.DictReader(ticket_info_line,
delimiter=’ ', skipinitialspace=True,
fieldnames=[‘startDate’, ‘startTime’,
‘expiresDate’, ‘expiresTime’, ‘servicePrincipal’])
if reader:
tgt_info = reader.next
if(check_if_ticket_is_for_this_domain(tgt_info, userAtDomain)):
if(check_if_ticket_is_still_usable(tgt_info)):
print “Ticket-granting ticket is still ok to use. Continuing…”
else:
print “Ticket-granting ticket is no longer usable, attempting to get a new one…”
self.cache_new_ticket(userAtDomain, password)
else:
print “Found a Ticket-granting ticket but not for %s. Continuing…” % (userAtDomain)

def run_kerberos_command(self, command_and_args, password):
try:
cmd = subprocess.Popen(command_and_args, stdin=PIPE, stdout=PIPE, stderr=PIPE)
if password:
cmd.stdin.write(‘%s\n’ % password)
cmd.wait()
except OSError:
raise errors.AnsibleError(“%s is not installed in %s. Please check you have installed krb-user and can run kinit from the command line” % (command_and_args[0], command_and_args[1]))
except IOError:
raise errors.AnsibleError(“could not authenticate as %s. Please check winrm group vars are correctly configured.” % (userAtDomain))

def cache_new_ticket(self, userAtDomain, password):
command_and_args = [‘kinit’, ‘%s’ % (userAtDomain) ]
self.run_kerberos_command(command_and_args, password)

def destroy_all_tickets(self):
command_and_args = [‘kdestroy’]
self.run_kerberos_command(command_and_args, None)

def set_ticket_cache(self):
ticket_cache = ‘/tmp/krb5cc_ansible-%s’ % (os.getpid())
print “Caching credentials to %s” % (ticket_cache)
os.environ[‘KRB5CCNAME’] = ticket_cache

the callbacks we use (on_setup and on_stats) are defined below:

def playbook_on_setup(self):
print “On setup. Checking to see if this run needs to join any windows/kerberos domains…”
self.set_ticket_cache()
hosts = self.playbook.inventory.get_hosts()
for host in hosts:
print “Considering host: %s” % (host.name)
host_vars = self.playbook.inventory.get_variables(host.name)
for var in host_vars:

TODO settle how to determine if a host needs to connect via kerberos.

perhaps having an ansible_ssh_user containing and ‘@’ or a '' would be enough

I don’t know, do local user names often get created containing @ or \ ?

if ( var == ‘ansible_authorization_type’ and host_vars[var] == ‘kerberos’):
print "Host %s is configured for windows/kerberos domain connection. Looking for ticket for %s " % (host.name, host_vars[‘ansible_ssh_user’])
self.find_or_acquire_ticket(host_vars[‘ansible_ssh_user’], host_vars[‘ansible_ssh_pass’])

def playbook_on_stats(self, stats):
print “On stats. Play tasks completed.”
print “Removing all tickets from credential cache: %s” % (os.environ[‘KRB5CCNAME’])

destroy tickets if configured to do so

#TODO make this optional based on ansible.cfg
self.destroy_all_tickets()

#end of plugin
`

12

In theory a username containing “@” or "" (e.g. a domain user) could still connect using basic authentication. There shouldn’t be any assumptions made on what auth mechanism to use based on the realm membership of the user. imho, of course.

13

Yes, its an edge case, but probably not a great idea to make it more difficult than it needs to be to get round it.

I suppose trying basic first and then kerberos might do it, but I’d want the successful connection type to be cached so ansible isn’t guessing every time winrm connection is established.

I will investigate and see if there is already a suitable mechanism in Ansible which could to used to convey how to auth for a given host. After all its really nothing more than an arbitrary piece of information about a host.

In the meantime a slightly debugged version of the ticket plugin I posted yesterday:

`

This plugin for ansible is intended to manage acquiring and destroying

kerberos or windows domain credentials.

import os
import json
import subprocess
import csv
import datetime
from datetime import datetime, timedelta
from subprocess import Popen, PIPE
from ansible import errors

class CallbackModule(object):
“”"
This callback module is intended to manage acquiring and disposing
of kerberos / windows domain credentials.
A new credential cache is created for each run and the contents
of the cache are destroyed when the run completes.
“”"

def init(self):
print “Windows/Kerberos Domain-joining plugin is active.”

TODO check kerberos user package has been installed here

and disable the plugin and warn use if so.

def days_hours_minutes(self, timedelta):
return timedelta.days, timedelta.seconds//3600, (timedelta.seconds//60)%60

def check_if_ticket_is_still_usable(self, tgt_info):
start = tgt_info[‘startDate’] + ’ ’ + tgt_info[‘startTime’]
expires = tgt_info[‘expiresDate’] + ’ ’ + tgt_info[‘expiresTime’]
ticket_valid_from = datetime.strptime(start, ‘%m/%d/%y %H:%M:%S’)
ticket_valid_to = datetime.strptime(expires, ‘%m/%d/%y %H:%M:%S’)
now = datetime.now()
ticket_aged_after= ticket_valid_to - timedelta(hours=2)
remaining_ticket_life = ticket_valid_to - now
if now > ticket_valid_to:
print “Ticket Expired”
return False
if now < ticket_valid_to and now > ticket_valid_from:
print “Ticket valid for %s days, %s hours, %s minutes” % (self.days_hours_minutes(remaining_ticket_life))
if now > ticket_aged_after:
print “Ticket is aged”
return False
return True

def check_if_ticket_is_for_this_domain(self, tgt_info, userAtDomain):
principle = tgt_info[‘servicePrincipal’]
parts = userAtDomain.split(‘@’, 2)
domain = parts[1].upper()
if ( principle.startswith(‘krbtgt’) and principle.endswith(domain) ):
return True
return False

def find_or_acquire_ticket(self, userAtDomain, password):
“”"

look for a ticket for configured ansible_ssh_user / ansible_ssh_password

todo open subprocesses (klist, kinit, kdestroy) the same way.

“”"
process = subprocess.Popen([‘klist’], stdout=subprocess.PIPE)
stdout, stderr = process.communicate()
stdout_lines = stdout.decode(‘ascii’).splitlines()
if not stdout_lines:
print "No credentials cache found. Attempting to get a ticket to cache… "
self.cache_new_ticket(userAtDomain, password)
return
else:
ticket_info_line =
ticket_info_line.append(stdout_lines[4]) # all the interesting info is on line 4
reader = csv.DictReader(ticket_info_line,
delimiter=’ ', skipinitialspace=True,
fieldnames=[‘startDate’, ‘startTime’,
‘expiresDate’, ‘expiresTime’, ‘servicePrincipal’])
if reader:
tgt_info = reader.next()
if(self.check_if_ticket_is_for_this_domain(tgt_info, userAtDomain)):
if(self.check_if_ticket_is_still_usable(tgt_info)):
print “Ticket-granting ticket is still ok to use. Continuing…”
else:
print “Ticket-granting ticket is no longer usable, attempting to get a new one…”
self.cache_new_ticket(userAtDomain, password)
else:
print “Found a Ticket-granting ticket but not for %s. Continuing…” % (userAtDomain)

def run_kerberos_command(self, command_and_args, password):
try:
cmd = subprocess.Popen(command_and_args, stdin=PIPE, stdout=PIPE, stderr=PIPE)
if password:
cmd.stdin.write(‘%s\n’ % password)
cmd.wait()
except OSError:
raise errors.AnsibleError(“%s is not installed in %s. Please check you have installed krb-user and can run kinit from the command line” % (command_and_args[0], command_and_args[1]))
except IOError:
raise errors.AnsibleError(“could not authenticate as %s. Please check winrm group vars are correctly configured.” % (userAtDomain))

def cache_new_ticket(self, userAtDomain, password):
command_and_args = [‘kinit’, ‘%s’ % (userAtDomain) ]
self.run_kerberos_command(command_and_args, password)

def destroy_all_tickets(self, credential_cache):
command_and_args = [‘kdestroy’]
self.run_kerberos_command(command_and_args, None)

def set_ticket_cache(self):
ticket_cache = ‘/tmp/krb5cc_ansible-%s’ % (os.getpid())
print “Caching credentials to %s” % (ticket_cache)
os.environ[‘KRB5CCNAME’] = ticket_cache

the callbacks we use (on_setup and on_stats) are defined below:

def playbook_on_setup(self):
print “On setup. Checking to see if this run needs to join any windows/kerberos domains…”
self.set_ticket_cache()
hosts = self.playbook.inventory.get_hosts()
for host in hosts:
print “Considering host: %s” % (host.name)
host_vars = self.playbook.inventory.get_variables(host.name)
for var in host_vars:

TODO settle how to determine if a host needs to connect via kerberos.

if ( var == ‘ansible_authorization_type’ and host_vars[var] == ‘kerberos’):
print "Host %s is configured for windows/kerberos domain connection. Looking for ticket for %s " % (host.name, host_vars[‘ansible_ssh_user’])
self.find_or_acquire_ticket(host_vars[‘ansible_ssh_user’], host_vars[‘ansible_ssh_pass’])

def playbook_on_stats(self, stats):
print “On stats. Play tasks completed.”
credential_cache = os.environ[‘KRB5CCNAME’]
print “Removing all tickets from credential cache: %s” % (credential_cache)

destroy tickets if configured to do so

#TODO make this optional based on ansible.cfg
self.destroy_all_tickets(credential_cache)

#end of plugin
`

14

Trod,

Actually, for security reasons a domain user cannot connect using basic authentication (even over TLS/SSL) I have tried. Check the documentation on the **WSManFlagUseBasic (**http://msdn.microsoft.com/en-us/library/aa384293(v=vs.85).aspx).

“Use Basic authentication. The client presents credentials in the form of a user name and password, directly transmitted in the request message. You can specify only credentials that identify a local administrator account on the remote computer.”

Domain authentication requires NTLM or Kerberos. If the username is a Windows style <user> then pragmatically, your only option is NTLM (check MIT Kerberos forums why). If you have a UPN then you can use NTLM or Kerberos.

I have worked with Microsoft Open Tech and implemented the only other mechanism, CredSSP over HTTP - but this still relies on NTLM or Kerberos to authenticate and establish and encrypted session.

15

Ian,

Thanks, that’s useful to know. I had assumed Trond meant you could have local adminstrators with names that contain @ or
I just tried creating a local admin account with the names ‘foo@bar’ and ‘foo\bar’ but on win 7 pro, it wouldn’t let me, (at least using the Users and Groups mechanism it wouldn’t).

Jon

16

So, I found I could just use a host var to tell my plugin to manage acquiring the tickets
by setting something like

[windows-servers:vars] use_ticket_plugin=true

in my inventory, but given what we now know about forms of usernames, it sounds like
using a convention will work.

Unless I’m mistaken, I think it be made to work as follows:

If ansible_ssh_user: contains @ and ansible_connection: winrm
then the winrm plugin should try kerberos, not basic auth

If your ansible controller is participating in domain, that’s fine, you have everything you need.

If your ansible controller isn’t participating in a domain, then you need to add the tickets plugin

The tickets plugin will look in each host’s host vars
If ansible_ssh_user: contains @ and ansible_connection: winrm
then it will attempt to acquire tickets.

Please point out any glaring omissions, otherwise I’ll work up new PRs for the changes to
the winrm connection plugin, the tickets callback plugin and documentation.

17

So the glaring omission in what I proposed above is that plugins aren’t used by ansible (as opposed to ansible-playbook). Not a huge problem perhaps but worth seeing if there is a way round it. I will investigate.

18

In latest dev, putting the following in your ansible.cfg will enable the plugins for bin/ansible:

bin_ansible_callbacks = True

Presumably this is False by default to maintain backwards compatibility though.

19

… but it seems to be reacting to different events, which is fair enough since at the moment I’m looking for on_start and on_stats.

20

Just to say I’ve had a go at getting my plugin to work with bin/ansible but with no success so far.
There’s an on_ok event which I could probably use to clean up the tickets, but I haven’t found an event analogous to ‘on_setup’ that I can use to work out what tickets need caching, so I’m working on adding an ‘on_matched_host’ event at the moment.