Consolidating redirects on docs.ansible.com

Project Discussions
, , ,
1

Hi everyone!

It seems like there has been a lot of discussion about consolidating things and reducing fragmentation in the community this year. In keeping with that theme, I’ve got some plans that I want to share with you.

Upcoming changes

The Ansible community team is going to modify URL redirects for plugins and module pages in Ansible 2.3, 2.4, 2.5, 2.6, 2.7, and 2.10 package docs.

We plan to merge the PR with these changes and consolidate the redirects after Cfgmgmtcamp takes place in February 2025. We’ll update this post with exact dates closer to the time.

Impact to community users

These changes affect only URLs to documentation for unmaintained and end-of-life (EOL) versions of the Ansible package.

The change for the user experience is that EOL pages for plugins and modules will redirect to a single revamped docs.ansible.com/collections.html page instead of the corresponding page under the collections subdirectory. Here are some specific examples:

https://docs.ansible.com/ansible/2.6/doas_module.html currently redirects to https://docs.ansible.com/ansible/latest/collections/community/general/doas_become.html

https://docs.ansible.com/ansible/2.4/lineinfile_module.html currently redirects to https://docs.ansible.com/ansible/latest/collections/ansible/builtin/lineinfile_module.html

After we make the changes to the redirects, both ansible/2.6/doas_module.html and ansible/2.4/lineinfile_module.html will redirect to a new docs.ansible.com/collections.html page.

This will also apply to some links in external sites that point to URLs such as https://docs.ansible.com/ansible/latest/modules/k8s_module.html

Instead of redirecting to ansible/latest/collections/community/kubernetes/k8s_module.html users will be redirected to the new collections.html page.

Wait. Doesn’t this change make things worse for users?

At first glance, you might be right in thinking that, yes, redirecting everything to the same page is not such a nice user experience. When you get redirected to that collections.html page, you need to do some work to find the page you’re looking for instead of being taken there directly.

However, there is a trade-off here that we think justifies the inconvenience. Let me explain.

Everything you need to know about redirects, but didn’t want to ask

The structure of content on docs.ansible.com has changed over the course of multiple releases. Before the split to collections and the Ansible package, pages for plugins and modules had a structure like this:

/ansible/2.3/copy_module.html
/ansible/2.3/ec2_group_facts_module.html
/ansible/2.3/find_module.html
/ansible/2.3/firewalld_module.html

Then it changed to a structure like this:

/ansible/2.5/modules/copy_module.html
/ansible/2.5/modules/ec2_group_facts_module.html
/ansible/2.5/modules/find_module.html
/ansible/2.5/modules/firewalld_module.html

And plugins were added and put into a structure like this:

/ansible/2.5/plugins/cache/redis.html
/ansible/2.5/plugins/callback/foreman.html

Then finally there were collections and the structure changed again to something like this:

/ansible/2.10/collections/ansible/builtin/copy_module.html
/ansible/2.10/collections/amazon/aws/ec2_group_facts_module.html
/ansible/2.10/collections/ansible/builtin/find_module.html
/ansible/2.10/collections/ansible/posix/firewalld_module.html
/ansible/2.10/collections/community/general/redis_cache.html
/ansible/2.10/collections/theforeman/foreman/foreman_callback.html

For each of those versions, the community docs folks created an .htaccess configuration file with redirect rules that would make sure anyone looking for those pages would get pointed to the latest collection instead.

You can find all the .htaccess configuration files in the ansible/docsite repository as follows:

Note that the ansible/11/.htaccess file is for any URLs in links from external sites that I mentioned earlier. For example, links to ansible/latest/modules/ get redirected to ansible/latest/collections/.

Benefits of making this change

If you look through all those .htaccess files, you’ll notice there are more than 24 thousand redirect rules in total. Yeah, it’s not 24k lines of code, but it’s still a lot of rules. And that comes with a cost that can be broken into two things:

  1. Lots of maintenance between releases. With each release of the Ansible package, collections get moved around and are either deprecated or removed. If we want to be serious about preventing 404s that degrade SEO authority for docs.ansible.com and to provide a good user experience, someone needs to update and validate the redirect rules.

Let’s look at the rule for that ansible/latest/modules/k8s_module.html path mentioned earlier. You can see the current rule here:

ansible/11/.htaccess#L2466

That module has moved and the target URL for the redirect returns a 404: https://docs.ansible.com/ansible/latest/collections/community/kubernetes/k8s_module.html

The redirect rule needs to be updated, so the target URL is this one: kubernetes.core.k8s module – Manage Kubernetes (K8s) objects — Ansible Community Documentation

  1. Keeping thousands of redirect rules defined in .htaccess files means we’re tied to Red Hat managed infrastructure for hosting docs.ansible.com. There’s nothing wrong with that, necessarily. It’s great that Red Hat takes on the financial cost and maintenance burden for that infrastructure to support the community. However, it limits the ability of community maintainers from handling docs publishing and getting full visibility and access to infrastructure. At the moment, only certain Red Hat employees can publish content to docs.ansible.com and make changes to the server-side. This sort of locked down environment strangles progress towards a more open hosting platform that offers lots more features.

We’ve been working on migrating docs.ansible.com to Read The Docs hosting and have talked about it on the forum for a while now. At the moment we have all the Red Hat managed projects, and some community projects, building under the ansible namespace at: https://ansible.readthedocs.io/

With a limit of 100 redirects per project we can’t recreate all those redirects on Read The Docs and migrate the docs.ansible.com subdomain.

So, to summarize the benefits of consolidating these redirect rules, we free ourselves from a massive maintenance overhead and make it possible to migrate to a platform with features like cross-project search and greater community access.

How do we plan to consolidate redirects?

We will remove all the .htaccess files in the ansible subdirectory of the ansible/docsite repository.

We will replace all the individual redirect rules with consolidated ones such as this:

RedirectMatch permanent "^/ansible/(devel|latest)/(plugins|modules)/(.+)\.html$" "/collections.html"

This rule matches URLs with ansible/devel/plugins and ansible/latest/plugins in the path and redirects them to docs.ansible.com/collections.html.

You can view the PR, and all of the conversation, with the consolidated rules here: https://github.com/ansible/docsite/pull/261

We are also updating the collections.html page to include some helpful links to make it easier for folks to find their way to the appropriate collection. You can find a PR preview build of that page here: collections.html preview build

Have all these consolidated redirects been tested?

Yes, we’ve tested these consolidated redirects thoroughly. You’re welcome to view our findings in this issue comment.

You can find more about the tests and the script used to check all the urls in this README

Call to Action: Give us your feedback

As always, we want to hear from you. Please reply to this post if you have any questions, suggestions, or comments. Feel free to also comment on the docsite PR directly. We’d appreciate any insights you might have or ways to improve our approach. Thanks!

4 Likes
2

I understand the problem, but at the spur of the moment I’m not sure if I like the solution.

People will be redirected somewhere else without getting an explanation. Wouldn’t it be better to redirect to a new page that explains things? Like, the module isn’t part of ansible (-core) anymore, and suggests to look at docs.ansible.com/collections.html. You could also mention that the module might not exist anymore, not even in a collection.

  1. I think this would be more helpful for users
  2. you could get rid of all those redirects, you only need one to a new page explaining things.

Or is this what you you’re planning to do and meant with “revamped”?

Sorry if I’m talking bullshit here. This is just the first thing that crossed my mind when I read your suggestion.

1 Like
3

Thanks for the feedback @mariolenz

Yes, this is what I mean by “revamped”. I’ve updated that collections.html page to try and explain things a bit more. I have the same concern in mind. That people will hit that page and not get a clear explanation.

Here’s a preview link to what I’ve got for that page now (pretty sure it can be improved): Ansible collections | Ansible Documentation

I figured updating that collections.html page makes sense because it’s already there but isn’t used for anything. It’s kind of an orphan page.

1 Like
4

Then your suggestion looks OK to me. Let’s see what others say or maybe find any flaws.

2 Likes
5

Hey everyone, bit of an update on this and I’d like to get some more feedback please. I’ll try and keep it concise.

I’ve updated the initial PR to consolidate redirects so that it applies to 2.x pages only. I did this for a couple of reasons. Firstly the user experience of pointing latest urls to the single docs.ansible.com/collections.html page seemed less than ideal. Secondly, I noticed that the rule for latest would have mistakenly redirected some of the existing plugins pages.

We now have a choice for how to handle latest urls for the plugins and modules paths.

Option 1: Consolidate the redirect rules in a similar way to the 2.x rules

I’ve opened a draft PR for that in the docsite repo.

The benefit of this approach is that it creates permanent 301 redirects and lowers the maintenance burden in the same way as the initial PR for 2.x rules.

The downside of this approach is that it means we will need to reproduce the redirects in the Read The Docs project side as part of the migration to that hosting platform. We’re also left with all the other latest|devel redirect rules in for pages that have moved during docs restructuring.

Another disadvantage is that this approach will mean the same user experience where latest/plugins and latest/modules pages get redirected to collections.html, which forces users to figure out where they need to go.

Option 2: Use a Sphinx extension to replace all latest redirect rules

I’ve opened a draft PR for that in the ansible-documentation repo.

The benefit of this approach is that we can maintain redirects directly within the Sphinx configuration. When moving to Read The Docs hosting, we won’t need to recreate redirect rules in the RTD project.

We can also provide a somewhat better user experience than redirecting to collections.html by redirecting users to the top-level collection page. For example, instead of redirecting modules/a10_server_module.html to the collections.html landing page, we can redirect to the collections/community/network/index.html page in the package docs.

As deprecated collections follow the lifecycle and get removed from the package, we can easily update the redirects in the project to point to that collections.html page too.

Finally this lets us handle all the latest and devel redirect rules in the same way, whether it is a redirect for the legacy plugins or modules pages or for pages that have been moved during a docs restructuring effort.

The main downside to the Sphinx extension is the fact that it adds *.html pages to the build output that return http 200 status instead of the 301 that is preferred for SEO. We can potentially mitigate that concern with a robots.txt configuration though.

Another disadvantage is that the Sphinx extension does generate a bunch of extra pages at build time. It doesn’t seem to impact performance or increase build time but it does add lots of files to the output.

I know I said I’d try to be concise but it feels like that was a lot of detail. Hopefully it all makes sense and is clear.

Thanks for all the feedback so far! Cheers.