A bit of history first - Back when it was just me and @acozine, we would hold all docs dependencies fixed for 6+ months and manually test everything after we updated it.
Now that there are more folks involved in the new docs repo (woohoo!!), we have some clever automation. But being the old-timer, I find it somewhat… well… scary… that dependencies (on devel branch mostly) update regularly. See this list of merged PRs for an example of what I’m talking about.
So my questions to the hive mind -
1 - Am I being too paranoid about dependency changes in the devel branch?
2 Should we have more stringent tests/validation for each PR before merging? (Right now, only @oranod and I can actually trigger a test publication of the docs and these PRs can get merged before we do that).
3 - Should we slow the pace ‘somehow’ so we are only doing automated update PRs once a month? (or perhaps that makes it even hard to test what’s changed to be sure it doesn’t break anything)?
As a final question as some of these fast dependency updates are being backported as well:
4 - What constraints should we put on the /latest/ docs with regard to dependency updates? In the past, we didn’t touch latest unless there was a bug. This was because of a combination of how hard it was to update, as well as …my general paranoia of don’t touch the site that thousands of people read every day.
@samccann I had similar thoughts yesterday while going through some PRs.
What if we make sure all dependency PRs are labelled and then have something similar to a triage session on Matrix? It shouldn’t take more than 10 minutes to go through things. Maybe that will slow things down a little but I feel like that could be beneficial for sharing and learning. And I expect we’ll soon hit a point where we’re all running at the same speed and can maybe relax those triage sessions.
Using more effective labels might help to make it easier to identify whether the dependency affects ci tooling (like the black formatter), docs build, transitive dependency and so on.
As far as the stringency of testing is concerned I feel like @gotmax23 has really done an amazing job with the noxfile. There’s so much more that we can do there but we’re on a path now that will enable much stricter checks for PRs, like converting warnings into errors etc. That gets us to the point where the RST is a lot cleaner and builds are more reliable. With some of the performance improvements that are apparently on the roadmap for Sphinx it could be of huge benefit to try and get as close to the latest as possible.
With 4) how about if we set up the testing site to build from the relaxed-requirements so we can observe any broken builds there before any major version bumps to the docs build stuff?
Ah thanks @gotmax23 for that reminder! I get confused with the auto-stuffs lol. So TL;DR; - there are two sets of auto dependencies - one for python linters and one for docs build. It’s the python linters that ‘those in the know’ are reviewing/merging as they come in and have no effect on the docs build. Docs dependency prs wait for me or @oranod to test on our staging server etc.
@oranod I do like the idea of an automated jenkins build that runs the relaxed requirements. Whether it publishes to docs.ansible.com or only runs on the internal server for now is fine.
And in general, I’m in favor of jumping to sphinx 7.2.x this week so it’s in place before the stable-2.16 branch pull next week. (aka so Ansible 9 will publish with the most recent sphinx)