Cloudflare prevents me from using docs.ansible.com sometimes

I’m often using the Ansible Community Documentation on https://docs.ansible.com/. This page seems to be using cloudflare to protect against overloading the website, which I can understand, especially due to people letting their LLMs crawl everything and clog the internet. However, those cloudflare pages are annoying, since the check often takes a long time – which may be caused by me sharing a public IP address with lots of other people (corporate network).

I’d like to have the docs experience without cloudflare getting in my way, e.g. by generating the docs locally.

Is there a way to have the documentation offline? I don’t necessarily need all docs of all modules, it would be sufficient to have them only for the ansible collections I have installed locally.

More than a decade ago, when the servers were down, people suggested using content from the git repo under devel/docsite, but that path doesn’t seem to be valid any more.

For python, many linux distributions package the python-docs or python3-docs package so you can have your full documentation on your local machine. Is there anything similar for ansible?

Oh, it seems like that content moved to a different git repo: GitHub - ansible/ansible-documentation: Ansible community documentation · GitHub
I can clone that and build it as described in the README.md.

hey @ChristianStadelmann yes, RST source was moved into the ansible/ansible-documentation repo a while back.

you should be able to build docs locally fairly easily following: Contributing to the Ansible Documentation — Ansible Community Documentation

the docs repo also contains a noxfile that makes it even easier: GitHub - ansible/ansible-documentation: Ansible community documentation · GitHub

you can also clone GitHub - ansible-community/package-doc-builds: This repository contains HTML and other assets from package doc builds from the ansible/ansible-documentation source repository. · GitHub and browse the generated HTML files directly if you want to skip building. those are the docs for the Ansible community package though, which includes ansible-core docs along with collection docs.

The instructions for building the docs locally don’t seem to work.

What I have:

1. A `Dockerfile` defining a container to build the docs in
FROM ubuntu:26.04

ENV DEBIAN_FRONTEND=noninteractive
ENV VIRTUAL_ENV=/opt/venv
ENV PATH="${VIRTUAL_ENV}/bin:${PATH}"

# Install the tools required to build the documentation.
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        python3 \
        python3-pip \
        python3-venv \
        make \
        git \
        ca-certificates \
    && rm -rf /var/lib/apt/lists/*

# Set up an isolated virtual environment for the build.
RUN python3 -m venv "${VIRTUAL_ENV}"

# Install the tested documentation dependencies.
# The requirements files are copied in first so this layer is cached
# independently of the rest of the repository.
COPY tests/requirements.in tests/constraints.in tests/requirements.txt /tmp/reqs/
RUN pip install -r /tmp/reqs/requirements.in -c /tmp/reqs/requirements.txt \
    && rm -rf /tmp/reqs

WORKDIR /workspace
2. A bash script facilitating the dockerfile
#!/usr/bin/env bash
set -euo pipefail

# Resolve the repository root (two levels up from this script).
REPO_ROOT="./ansible-documentation"
DOCS_OUTDIR="./output/"

IMAGE_NAME="ansible-docs-builder"
CONTAINER_WORKDIR="/workspace"
CONTAINER_DOCSDIR="/out"
DOCKERFILE="./Dockerfile"
REPO_URL="https://github.com/ansible/ansible-documentation.git"

# Build the image only if it does not already exist.
# Force a rebuild with: FORCE_REBUILD=1 ./build_ansible_docs.sh
if [[ "${FORCE_REBUILD:-0}" == "1" ]] || ! docker image inspect "${IMAGE_NAME}" >/dev/null 2>&1; then
  echo "Building Docker image: ${IMAGE_NAME}"
  docker build -f "${DOCKERFILE}" -t "${IMAGE_NAME}" "${REPO_ROOT}"
else
  echo "Reusing existing Docker image: ${IMAGE_NAME}"
fi

######################## Prepare git repo ###########################
# Fetch the latest commits and switch to the latest stable branch.
echo "Cloning repo and fetching latest commits in: ${REPO_ROOT}"
if [[ ! -d "${REPO_ROOT}/.git" ]]; then
  echo "Cloning repository into: ${REPO_ROOT}"
  git clone "${REPO_URL}" "${REPO_ROOT}"
fi
git -C "${REPO_ROOT}" clean -xdf
git -C "${REPO_ROOT}" fetch --all --prune

# Determine the latest stable branch by parsing remote branches named
# "stable-<major>.<minor>" and selecting the highest version.
LATEST_STABLE_BRANCH="$(
  git -C "${REPO_ROOT}" branch --list --remote 'origin/stable-*' \
    | sed 's#^[[:space:]]*origin/##' \
    | grep -E '^stable-[0-9]+\.[0-9]+$' \
    | sort -t- -k2 -V \
    | tail -n1
)"

if [[ -z "${LATEST_STABLE_BRANCH}" ]]; then
  echo "Error: could not determine the latest stable branch." >&2
  exit 1
fi

echo "Switching to latest stable branch: ${LATEST_STABLE_BRANCH}"
git -C "${REPO_ROOT}" switch "${LATEST_STABLE_BRANCH}"
git -C "${REPO_ROOT}" pull --ff-only

######################## Finished preparing git repo ###########################

# Ensure the output directory exists so it is owned by the current user,
# not created as root by Docker.
mkdir -p "${DOCS_OUTDIR}"

echo "Building Ansible documentation."
echo "Mounting repository: ${REPO_ROOT}"
# docker run --rm -it --user "1000:1000" --volume "../ansible-documentation:/workspace" --volume "/out:./output" --workdir "/workspace" --env HOME=/tmp "ansible-docs-builder"
docker run --rm -it \
  --user "$(id -u):$(id -g)" \
  --volume "${REPO_ROOT}:${CONTAINER_WORKDIR}" \
  --volume "${DOCS_OUTDIR}:${CONTAINER_DOCSDIR}" \
  --workdir "${CONTAINER_WORKDIR}" \
  --env HOME=/tmp \
  "${IMAGE_NAME}" \
  bash -c '
    set -euo pipefail

    # Clone the required parts of Ansible Core for the docs build.
    python3 docs/bin/clone-core.py

    # Build the module documentation plus all rST pages.
    make -C docs/docsite coredocs
    #bash
  '

echo "Documentation build complete."

This fails, though:

$ ./build.sh
Reusing existing Docker image: ansible-docs-builder
Cloning repo and fetching latest commits in: ./ansible-documentation
Switching to latest stable branch: stable-2.21
Already on 'stable-2.21'
Your branch is up to date with 'origin/stable-2.21'.
Already up to date.
Building Ansible documentation.
Mounting repository: ./ansible-documentation
Cloning into '/tmp/tmp2ruxjmhu'...
remote: Enumerating objects: 7693, done.
remote: Counting objects: 100% (7693/7693), done.
remote: Compressing objects: 100% (5411/5411), done.
remote: Total 7693 (delta 604), reused 5391 (delta 504), pack-reused 0 (from 0)
Receiving objects: 100% (7693/7693), 4.13 MiB | 12.13 MiB/s, done.
Resolving deltas: 100% (604/604), done.
Updating files: 100% (5870/5870), done.
Updating 'bin' ...
Updating 'lib' ...
Updating 'packaging' ...
Updating 'test/lib' ...
Updating 'MANIFEST.in' ...
Updating 'pyproject.toml' ...
Updating 'requirements.txt' ...
make: Entering directory '/workspace/docs/docsite'
Creating symlinks in core_structure
ln -sf ../rst/core_index.rst rst/index.rst
ln -sf ../dev_guide/core_index.rst rst/dev_guide/index.rst
../../hacking/build-ansible.py collection-meta --template-file=../templates/collections_galaxy_meta.rst.j2 --output-dir=rst/dev_guide/  ../../lib/ansible/galaxy/data/collections_galaxy_meta.yml
../../hacking/build-ansible.py document-config --template-file=../templates/config.rst.j2 --output-dir=rst/reference_appendices/  ../../lib/ansible/config/base.yml
../../packaging/cli-doc/build.py rst --output-dir=rst/cli/
../../hacking/build-ansible.py document-keywords --template-dir=../templates --output-dir=rst/reference_appendices/ ../../lib/ansible/keyword_desc.yml
../../hacking/build-ansible.py docs-build core -o rst
Running ['antsibull-docs', 'stable', '--deps-file', '/tmp/tmpxs6ycemq/ansible.deps', '--ansible-core-source', '/workspace', '--dest-dir', 'rst']:
Traceback (most recent call last):
  File "/workspace/docs/docsite/../../hacking/build-ansible.py", line 130, in <module>
    main()
    ~~~~^^
  File "/workspace/docs/docsite/../../hacking/build-ansible.py", line 119, in main
    retval = command.main(args)
  File "/workspace/hacking/build_library/build_ansible/command_plugins/docs_build.py", line 275, in main
    return generate_core_docs(args)
  File "/workspace/hacking/build_library/build_ansible/command_plugins/docs_build.py", line 147, in generate_core_docs
    return antsibull_docs.run(full_command)
           ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.14/site-packages/antsibull_docs/cli/antsibull_docs.py", line 959, in run
    return ARGS_MAP[parsed_args.command]()()
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/opt/venv/lib/python3.14/site-packages/antsibull_docs/cli/doc_commands/stable.py", line 139, in generate_docs
    collection_tarballs = asyncio.run(
        retrieve(
    ...<7 lines>...
        )
    )
  File "/usr/lib/python3.14/asyncio/runners.py", line 204, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/usr/lib/python3.14/asyncio/runners.py", line 127, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/lib/python3.14/asyncio/base_events.py", line 719, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/opt/venv/lib/python3.14/site-packages/antsibull_docs/cli/doc_commands/stable.py", line 90, in retrieve
    responses = await asyncio.gather(*requestors.values())
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.14/site-packages/asyncio_pool/base_pool.py", line 108, in _wrap
    res = await coro
          ^^^^^^^^^^
  File "/opt/venv/lib/python3.14/site-packages/antsibull_core/ansible_core.py", line 392, in get_ansible_core
    install_file = await pypi_client.retrieve(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        ansible_core_pypi_version.public, tmpdir
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/venv/lib/python3.14/site-packages/antsibull_core/ansible_core.py", line 161, in retrieve
    for release in release_info[ansible_core_version]:
                   ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
KeyError: '2.21.2.post0'
make: *** [Makefile:189: core_plugins] Error 1
make: Leaving directory '/workspace/docs/docsite'

Am I doing something wrong, or is this a bug somewhere?

I think you’re missing the step to clone the ansible-core repo with: ansible-documentation/docs/bin/clone-core.py at devel · ansible/ansible-documentation · GitHub

Thanks for the helpful comments!

I tried and failed (see my comment above that I wrote at the same time as you)

This repo is like 20GB in size. I guess cloning the whole repo would be overkill for my purposes. Cloning with --depth=10 works fine though, and I guess I’ll go with that.

No, that’s in the bash script.

right you are. that’s my bad. I can see it with:

# Clone the required parts of Ansible Core for the docs build.
    python3 docs/bin/clone-core.py

I guess I just looked quickly and didn’t see it at first. yeah that package-docs-repo is huge and cloning it is probably not ideal.

I’ll take a closer look later and see if I can work out the issue with building docs locally anyway.

You just need the latest version, so --depth=1 is also fine.

Your issue is caused by a breaking change in Python’s ast module in Python 3.14.

The s attribute of ast.Constant was deprecated in Python 3.12, and removed in Python 3.14. The code that checks the version of the ansible-core checkout uses ast.Constant().s to extract the version, and now fails with Python 3.14.

I’ll create a PR to fix this in antsibull-core.

(In other words: for now you have to stick to Python < 3.14 to build the docs.)

Fix: Rewrite ansible-core version extraction and fix bug with Python 3.14 by felixfontein · Pull Request #202 · ansible-community/antsibull-core · GitHub

Thanks for looking at this @felixfontein I was poking at it in the background as well and was wondering if it might have been the case that the docs build was happening on stable-2.21 instead of devel. I guess not.

It also seems like this affects the coredocs target nox -s make fails but nox -s make -- webdocs and nox -s make -- singlehtml succeed if that’s any help. Cheers.

That’s because only the coredocs target ends up using the problematic code path :slight_smile: