(developers) All about the new auto-generated docs feature, how it works, and what I need your help with

[Michael_DeHaan]

Various module sources now contain a DOCUMENTATION doc string that
contains a YAML description of the help of that module.

This doc string is used to generate the ReStructuredText source code
to the modules page of the docs site, for modules that have them.

This feature is mostly due to all the hard work of JP Mens (huge
thanks), though I have had to tweak it some and have as a result
probably broken a few of his other exciting templates it can also use.

This include being able to generate manpage documentation for modules
as well as a LaTex-formatted PDF. Both of these are awesome ideas
and I think he's only going to have to make some minor tweaks to the
templates to get that working again.

The next steps are to apply the YAML "DOCUMENTATION" doc string to
every single module that needs it. The module docs text used to
describe each option must mirror the existing documentation page
content.

One this is done (including for all action modules, which will be
stubs that contain NOTHING but a documentation string) the web site
documentation for modules can easily be maintained by module authors
WITHOUT having to make seperate changes in the ansible-docs project.

This opens the way for other things, such as a really kick-ass module
docs viewer that @mavimo prototyped -- http://agavee.com/ansible-docs/
-- which we could possibly include in something like
Ansible-commander.

It also will ultimately allow ansible itself to introspect module
arguments prior to dispatch to remote hosts, allowing it do new error
checking on remote hosts. (The implementation of this is somewhat
theoretical and not on any immediate roadmap, but sounds cool).

So, I guess the conclusion is this -- let's get to work annotating
modules with DOCUMENTATION strings! Look at ini_file (It's new!) for
a relatively pristine example of what it looks like.

Any questions please let me know, and once again, thanks to JP Mens
for both the idea and hard work in getting it working!

I'm not sure how to best divide up the module documentation work, so
how about replying to this thread with what modules you are working on
docs for and everybody takes three or four at a time? We'll be done
pretty quick.

--Michael

[jpmens1]

docs for and everybody takes three or four at a time? We'll be done
pretty quick.

Starting on:

        - assemble
        - authorized_key
        - command
        - copy

now.

        -JP

[jpmens1]

        - assemble
        - authorized_key
        - command
        - copy

Done.

It's no great fun (copy/paste & all), but there are a few things you
want to check for:

1. Tabs in the YAML DOCUMENTATION string break the formatter; a warning
   diagnostic that the module "could not be parsed" is issued.
2. I recommand you take a module, add the DOCUMENTATION string and then
   run the formatter on it, specifying just that one module with -m and
   the type (-t rst):

   ./hacking/module_formatter.py -t rst -m copy

   If the program completes w/o errors, you're reasonably certain of
   things being ok.

3. A `make manpages' in the top-level Ansible directory produces man
   pages in docs/man/man1/*.man

Regards,

        -JP

PS: Pull-Request submitted, of course

[Stephen_Fromm]

I’ll take git, user, group, and service as a starting point.

sf

[Marco_Vito_Moscarito (Marco Vito Moscaritolo)]

Started:

- apt
- apt_repository

Bye
       Marco

[jpmens1]

It's no great fun (copy/paste & all), but there are a few things you
want to check for:

1. Tabs in the YAML DOCUMENTATION string break the formatter; a warning
   diagnostic that the module "could not be parsed" is issued.

Forgot to mention another thing: if you copy paste from the existing
modules.html, beware apostrophies and other non-ASCII characters: you'll
have to replace those please.

        -JP

[jpmens1]

Doing:
        - template
        - wait_for
        - yum

[Tim_Bielawa]

I'll get pause and nagios now.

[Marco_Vito_Moscarito (Marco Vito Moscaritolo)]

Status report (public, all can edit)

* https://docs.google.com/spreadsheet/ccc?key=0Aq4q7nXKMUBzdGt1dWFmWTFwNGlBLWt3LU1UNGo1MWc#gid=0

I'm start to move this list+issue information in docs.

Bye
       Marco

[Marco_Vito_Moscarito (Marco Vito Moscaritolo)]

Some update on autogenerated docs.

I write a small script to merge each single JS file all generated
module.json doc files.

Rewrite the UI to respect the new information available on DOCUMENTATION string.

You can see the preview from:

* http://agavee.com/ansible-docs/

Next step will to parse U / M / B / I command from docs and some UI /
graphic improvements.

If you have some feedback.. you're welcome!

Bye
       Marco

[jpmens1]

If you have some feedback.. you're welcome!

Feedback? This is absolutely beautiful, Marco. Very useful. I think
we're doing this right.

        -JP

[jpmens1]

Sunday morning (CET) here: Doing:

  - facter
  - easy_install
  - fetch
  - mount
  - ohai

Notes:
  - If you need to use a colon ( in a description, example, etc.
    please quote the whole line. Eg:

        - description:
            - "does this because: that"

I've taken all currently submitted DOCUMENTATION strings and fixing
little things like that, so don't you go back to do that! I greatly
appreciate the work you've all invested in this so far. I believe this
is going to rock. Have you seen Marco's live docs site? It's pretty WOW!

        -JP

[jpmens1]

- virt also completed and pushed.

Just seven more to go, guys: who'll chip in? These are still marked
as open on [1]:

        - async_status
        - async_wrapper
        - fireball (Michael: this one's for you!
        - seboolean
        - sexlinux
        - slurp

Regards,

        -JP

[1] https://docs.google.com/spreadsheet/ccc?key=0Aq4q7nXKMUBzdGt1dWFmWTFwNGlBLWt3LU1UNGo1MWc#gid=0

[jpmens1]

        - sexlinux

Yeah, ok, for all the stuff I've been doing on Ansible, probably just
wishful thinking here...

[Michael_DeHaan]

Awesome! (I mean getting the modules down, of course!). Thanks everyone!

There's no need to document async_wrapper, BTW, you would never
directly call it.

(You can directly call async_status, but I would only say you'd use
that if you launched something with async and forgot to poll.)

[Michael_DeHaan]

I already replied on twitter, but this is amazing stuff.

As you suggested, we'll probably keep the static version around for
Google juice but I'm definitely going to make this very prominent on
the main docs page.

Can you send a pull request to make it part of the
ansible/ansible-docs repo (and modify the Makefile) so it can be built
every time?

It would also be nice if we could tweak it to match the ansible-docs
header, minus probably the Chapter/Page and search links

     http://ansible.cc/docs/

[Marco_Vito_Moscarito (Marco Vito Moscaritolo)]

I already replied on twitter, but this is amazing stuff.

Thanks

As you suggested, we'll probably keep the static version around for
Google juice but I'm definitely going to make this very prominent on
the main docs page.

Maybe we can genrate static HTML version _AND_ HTML/JS version.

Can you send a pull request to make it part of the
ansible/ansible-docs repo (and modify the Makefile) so it can be built
every time?

I'm starting to create a script to add as WebHook in github to rebuild
docs, I'm send you a provate message to indicate specific URL to call.

It would also be nice if we could tweak it to match the ansible-docs
header, minus probably the Chapter/Page and search links

     http://ansible.cc/docs/

I need to complete web interface/build procedure before having a pull
request, but I'll do it ASAP

       Marco

[Michael_DeHaan]

I already replied on twitter, but this is amazing stuff.

Thanks

As you suggested, we'll probably keep the static version around for
Google juice but I'm definitely going to make this very prominent on
the main docs page.

Maybe we can genrate static HTML version _AND_ HTML/JS version.

This is what I'm suggesting, yes.

Can you send a pull request to make it part of the
ansible/ansible-docs repo (and modify the Makefile) so it can be built
every time?

I'm starting to create a script to add as WebHook in github to rebuild
docs, I'm send you a provate message to indicate specific URL to call.

Hmm, you mean rebuilding it on your servers? I'm confused as to why
this would be necessary.

Things really must live in the main project where anybody can rebuild
and patch it.

[Marco_Vito_Moscarito (Marco Vito Moscaritolo)]

Can you send a pull request to make it part of the
ansible/ansible-docs repo (and modify the Makefile) so it can be built
every time?

I'm starting to create a script to add as WebHook in github to rebuild
docs, I'm send you a provate message to indicate specific URL to call.

Hmm, you mean rebuilding it on your servers? I'm confused as to why
this would be necessary.

Not only on my server, but on all server that would generate docs locally.

Things really must live in the main project where anybody can rebuild
and patch it.

The module docs (only docs for module, not all docs!) live ansible
repository, is only a copy of
http://github.com/ansible/ansible/libraries/* file with a data
manipulation, so I tink is not a great solution have a different copy
of it in a different repository.

In docs repo can be add all require script to genrate docs and base
source file (HTML, js app files, custom CSS, ..)

What I immagine is:
* A web hook in github generate a request to some serve (I'm writing
it on my server, but I think you can generate it on your server, in
heroku, ...).
* When webhook is called start a git pull command and copy data on
local machine;this server generate JSON docs and aggregate it on
modules.js (maybe heroku can't be used to do this, it don't support
local file writing).
* This server can move file in some external server (on Google CDN,
or in a ansible.cc server)
* User require appplication data (modules.js) from server with generated data.

Let me know what do you think about it.

Bye
       Marco

[Michael_DeHaan]

What I immagine is:
* A web hook in github generate a request to some serve (I'm writing
it on my server, but I think you can generate it on your server, in
heroku, ...).
* When webhook is called start a git pull command and copy data on
local machine;this server generate JSON docs and aggregate it on
modules.js (maybe heroku can't be used to do this, it don't support
local file writing).
* This server can move file in some external server (on Google CDN,
or in a ansible.cc server)
* User require appplication data (modules.js) from server with generated data.

Let me know what do you think about it.

I think it's unnecessary and way too much to maintain and think about.

All we need is a Makefile target in the docs repo, I have to do a
build of the docs site periodically anyway.

We basically have this already, it's "make docs". It would have to
call the module formatter once to output the JSON
file. Super easy.

I'm more than happy to continue hosting docs on ansible.cc.

--Michael