ansible-commander update: data layer complete, REST in work, next steps...

Archives Ansible Project
1

I kind of feel like I'm giving kickstarter updates on something that
isn't done yet, but I want you to be excited :slight_smile:

The ansible-commander data layer can now do everything I want to do
with groups and hosts, and is ready to support wiring things up to
REST now!

I'm going to be writing a very very basic and dumb test script,
probably using Requests, to exercise the wired up API, to prove we can
do the following things via JSON/REST:

* add a user
* change the user's password
* delete a user
* add a group
* change the group name, or change the group's parents
* change the group variables
* add a host
* change the groups the host is directly in
* change the host variables
* ask for ansible, what groups contain what hosts
* ask for ansible what the 'blended' variables to use for the host
are, in the inventory script

If you look at commander.py you can see the URL routes are already
layed out (but may change somewhat).

Once that is done, we write the inventory plugin, which should be
super simple, which just talks to the REST API in
unauthenticated fashion. This will allow multiple ansible servers
(should you want that) to live on different machines and all talk to
the central inventory server. This will be a really short script as
it's not doing much more than what curl would do!

Next steps? Stage2!

We are then basically fully complete for the *BACKEND* components of
what will release for Ansible-Commander 0.1, and you'll be able to go
ahead and use it however you want -- and the API won't have to evolve
much more at all for what it can do now. (I anticipate this will be a
really good way to write integrations with other inventory systems,
like if you had a proprietary CMDB and wanted to sync it with
Ansible's information.)

This is to say, the front end components then need to be written, and
we'll start on that next.

I anticipate there's less than a day's worth of work (in man-hours)
left on the backend parts before the REST API and inventory plugin
will be done.

The data layer was hideously complicated to handle the logic of things
like "what if I delete a group that has both parents and child groups,
etc" -- decisions previously humans had to handle when working with
the INI files -- but it's all out of the way now, which is good -- and
it works pretty nicely.

There is a stage 3 -- playbook logging and visualization, and so
forth, but this will probably come in later releases of
Ansible-commander.

Coming along nicely. Just wanted to give you an update, there should
be a functional REST impelementation to hammer on very soon!

--Michael

2

Lookin good, Michael. Take this with a grain of salt, but Flask has a good test client built into it if you want to avoid adding another dependency for testing. It looks like:

client = app.test_client()
response = client.post(‘/api/…’, data=“{ … json data … }”, content_type=“application/json”)
assert “something” in r.data

3

Lookin good, Michael. Take this with a grain of salt, but Flask has a good
test client built into it if you want to avoid adding another dependency for
testing. It looks like:

   client = app.test_client()
   response = client.post('/api/...', data="{ ... json data ... }",
content_type="application/json")
   assert "something" in r.data

Excellent! Very good to know, I was hoping it might have something like that.

(Another advantage of going this way is that If we also call
acom_data.test_mode() we can make it use the testing database, meaning
it's safe to develop
on a machine you also are using in real life)

4

Just pushed code to enable the users API to work via REST.

https://github.com/ansible/ansible-commander/blob/master/commander.py

It's uber-trivial from here to wire up groups and users, though I want
to make sure there are some solid tests before turning people lose on
it, so I'll get that in over the next few days.

Huge thanks to Matt for pointing out the test class -- it is working great:

https://github.com/ansible/ansible-commander/blob/master/test.py

If folks are interested in helping develop on this, it may be worth
some of your time to see if you can get it set up and successfully
call an API method.

first install postgresql-server and python-flask
./setup.sh will launch a rather simple ansible playbook to configure
the database
then ./commander.py to run Ansible commander on port 5000 with Flask

curl -u 'admin:gateisdown' http://127.0.0.1/api/users/ --verbose

and you should see a user automatically created, that you can then
edit by posting the user record back at it, if you so like.

Everything takes JSON, so if you want to change the password it is just:

PUT to api/v1/users/admin:

{ 'user' : 'admin', '_password' : 'newpassword' }

You can see an 'href' in any of the REST elements that come back that
show you what URL to use to manipulate them.

By convention, any time you see a field name that doesn't start with
an underscore, it's public and you can edit things with PUT. If it
starts with an underscore, it might be protected data (usually is), or
in the case of password, it's actually a write only variable.

While it may not be strictly pure, PUT allows partial edits. If you
just want to change part of a resource, you can post only the fields
you want to change. By convention, the UI will likely post everything
when editing resources.

Once this is more established there will definitely be a page on the
site about ansible-commander including an API guide.

5

Hi Mike…here are few things to consider when developing an API with Flask. And as always, take it with a grain of salt.

Versioned API endpoints
Its generally good practice to add a version number you the endpoints. For instance, the first version should perhaps be prefixed with /api/v1. This will make future improvements easier to integrate and perhaps allow for third party clients/front ends to upgrade without breaking their UI code. Each API could be developed as a Flask Blueprint.

App Namespace
Flask plays nicely with “packaged” apps. I would simply move commander.py into the acom package, perhaps into the __init__.py file or something like core.py. This would allow the app to be more flexible in other contexts, should someone decide to integrate it with additional apps. A root level file named runserver.py or wsgi.py with the following code would replace commander.py at the root with:

from acom import app as application

if name == ‘main’:
application.debug = DEBUG
application.run(debug=DEBUG)

This file can then be used by Apache’s mod_wsgi or another application container such as Gunicorn or uWSGI which is often used with Nginx.

Flask app config file
An additional config file for the Flask app could be useful. It can be loaded as such:

app.config.from_envvar(‘ACOM_CONFIG_FILE’)

Where ACOM_CONFIG_FILE is the full path to a Flask compatible config file.

At any rate, just a few thoughts that may or may not be useful.

6

Hi Mike...here are few things to consider when developing an API with Flask.
And as always, take it with a grain of salt.

Versioned API endpoints
Its generally good practice to add a version number you the endpoints. For
instance, the first version should perhaps be prefixed with `/api/v1`. This
will make future improvements easier to integrate and perhaps allow for
third party clients/front ends to upgrade without breaking their UI code.
Each API could be developed as a Flask Blueprint.

Note -- I develop non-trivial REST APIs for a living. While it's
socially acceptable 'best practices', the truth is versioned APIs are
kind of misleading --a better approach is to version REST APIs exactly
like you would version a stable Python API -- which is we're only
adding fields and URLs/methods, and not taking fields or URLs/methods
away. It takes more planning up front, and a commitment to have
defaults for new fields. In a systems-management application, it's
super unlikely that you want to maintain an older version of your
business logic parallel in the tree, so a versioned API is only
versioning the veneer on the front. Further, we release frequently
enough that even a one-or-two versions back policy on API versions is
not enough.

What I am going to do is provide API discovery, such that if you grab
'/' you can see where the various services are, and that '/' node can
include the version number of the app, and we can include docs about
things that are going to be deprecated. Thus it's easy to see what
top level URLs are available. If resources start to hang off the
main resource, these will also be discoverable. However, though
while I'm doing this, I'm *NOT* going to rewriting the URLs, they
should be predictable and safe to hard code for people writing simple
scripts.

A lot of folks are going to be writing scripts against this, and it's
important that they *not* need to follow Ansible closely to know that
the interface is stable and is going to remain working. It's all
about lowering barriers to entry and keeping those scripts stable.

In other words, I'm making it hard to make incompatible changes on purpose.

App Namespace
Flask plays nicely with "packaged" apps. I would simply move `commander.py`
into the `acom` package, perhaps into the `__init__.py` file or something
like `core.py`. This would allow the app to be more flexible in other
contexts, should someone decide to integrate it with additional apps. A root
level file named `runserver.py` or `wsgi.py` with the following code would
replace `commander.py` at the root with:

It's just Python, so there's no reason why it shouldn't :slight_smile:

What we have now is just a starting point.

    app.config.from_envvar('ACOM_CONFIG_FILE')

Currently ansible reads from /etc/ansible/commander.cfg, though as
some of those other settings are useful, that's probably reasonable.

7

Got it. Appreciate the thorough explanation!

8

I haven't yet had a chance to look through this yet, and Github is
returning 502 errors for me right now, so forgive me if this is an
FAQ, but are there any plans to wire this up to an external
authentication provider, like LDAP, radius, active directory,
container-provided basic auth, or even local accounts?

* Michael DeHaan <michael.dehaan at gmail.com> [2012/09/10 22:19]:

9

No plans for me to do it in the short term (maybe longer term,
depending on what's in queue), but I would gladly take patches for
this, and they aren't hard. There is some ldap auth code we can
likely learn from in Cobbler, and it could
read the authentication mode from the config file.

It could drop it pretty nicely in the "authenticate" method and
completely bypass the users API.