Testing MkDocs documentation examples

The short answer

Test an MkDocs site in two complementary ways: build the site with its own checks, then run the examples a reader would follow in a clean environment. Start with the published installation path, keep dependent blocks in page order, and compare any promised output. Record the alternatives, included content and generated API examples that your test did not cover.

mkdocs build --strict builds the documentation and aborts on warnings; it does not establish that a code example works. mkdocs get-deps shows required PyPI packages inferred from plugins in mkdocs.yml; it does not establish the dependencies of every example. See the MkDocs command-line reference for both commands. The broader documentation testing guide explains how execution fits alongside other checks.

DocsAlive detects MkDocs from mkdocs.yml, extracts fenced blocks, and plans supported examples for execution or another stated treatment. It recognizes Material tabs as alternatives and picks one variant from each tab group. It runs supported steps in page order in a fresh container per page and classifies results. A passing build and a passing example check answer different questions; use both when both matter.

Where MkDocs examples live

An MkDocs project commonly has a mkdocs.yml configuration file and a documentation directory containing Markdown pages. The docs_dir setting identifies the documentation directory, while nav can arrange pages and sections for readers. If nav is omitted, MkDocs creates a nested navigation list from Markdown files under docs_dir. Navigation can also contain links outside the documentation files. These behaviors are described in the MkDocs configuration reference.

That arrangement matters for coverage. A tester who searches only the pages named in a hand-written navigation list may miss a Markdown file that is still part of the site. A tester who assumes every navigation item is a local page may also try to treat an external destination as a repository file. Start by identifying the actual documentation directory and the pages intended for the build. Then compare that inventory with the set of pages your example test examined.

Keep the source location in the test report. A reader sees a page, but a maintainer needs to know which file and line to change. A page may also assemble material from elsewhere or generate reference content from code. In those cases the page file is a useful entry point, but it may not be the place where the example itself is authored.

The static-site documentation testing guide covers the broader distinction between source, build and published site. For a repository landing page outside the MkDocs tree, add a separate README check. Treat examples in a README as another reader entry point rather than assuming the site build covers them.

Configure the features your pages use

This illustrative mkdocs.yml names a docs directory and a navigation item:

site_name: Acme documentation
docs_dir: docs
nav:
  - Getting started: index.md

Tabbed examples like the one below also need the Markdown add-ons that the Material content tabs reference lists for mkdocs.yml (SuperFences and Tabbed). Copy those lines from that reference rather than from an old page.

The configuration establishes how the site processes the Markdown; it is not a test of the commands in that Markdown. Keep a site's real configuration and dependency setup under review as the site changes. When a new extension or plugin changes what readers see, revisit the example inventory and the test method as well.

Use the build as a separate signal

Run mkdocs build --strict against the configuration used for the site. The CLI reference says that strict mode causes the build to abort on any warnings. A successful strict build is useful evidence that the build completed without warnings under that configuration. The meaning of individual diagnostics depends on the site's settings; the configuration reference describes settings for navigation and link validation.

A successful build does not show that a package exists, that a shell command completes, or that an import matches the installed library. Those require a test that actually follows the relevant example. Conversely, a working example does not prove that the site builds or that its navigation and links are right. Keep the results distinct so a reviewer can see which promise each check addresses.

mkdocs get-deps has a narrower purpose: it shows required PyPI packages inferred from plugins in mkdocs.yml. Use its output to investigate the site build environment, not as an inventory of all packages referenced in code samples. A tutorial can install or import a package that is unrelated to an MkDocs plugin. It can also require a system tool, a file, or a service that cannot be inferred from a plugin list.

When a strict build fails, inspect the warning and the configuration before changing a page. When an example fails, reproduce the example with its documented prerequisites before changing the site configuration. Both results deserve attention, but they have different owners and different fixes.

Treat Material tabs as reader choices

Material for MkDocs code blocks supports presentation features for code examples. Content tabs let a page group alternative content, and SuperFences enables nested content in tabs. Presentation can make a page easier to use, but the test still needs to decide which content represents a runnable path.

For example, a page might offer two ways to install the same fictional package. The following is Markdown source for such a page; the package name is illustrative and is not a promise that a real package exists:

# Install the demo

Choose one installation method in your prepared Python environment.

=== "pip"

    ```bash
    python -m pip install acme-docs-demo
    ```

=== "uv"

    ```bash
    uv pip install acme-docs-demo
    ```

A reader chooses a tab. Running both tabs consecutively in one environment would exercise a different journey and could let the first installation affect the second. Test each important alternative from an equivalent clean starting state. State which path was tested and what was available before it began. The example assumes the chosen installer is already available; the real page should tell readers how to meet that prerequisite.

DocsAlive recognizes tabbed code groups as alternatives and picks one variant from each group. Other variants are skipped as alt-variant. One successful run therefore does not establish that all package-manager or language tabs work. Plan additional coverage for the alternatives your readers use, and inspect the report to see which path was selected and which blocks were skipped.

Tab labels are not the whole test plan. Check that the content under each label is a coherent alternative, that later instructions make sense after the selected path, and that the page does not quietly require a tool it never introduces. If a later block depends on a file or command produced in one tab but not another, test each path through that point. For installation-specific cases, use the installation testing guide.

Account for included and generated content

The Material code blocks reference documents embedding external files into code blocks with Snippets, and the content tabs reference uses SuperFences to nest code blocks inside tabs. This means the text in the page source and the example presented to a reader can differ.

For each included example, identify both the page that presents it and the file that supplies it. Test the content readers actually see, in the context the page supplies. Then report the underlying source location so the owner can fix the right file. If an included example appears on several pages, check whether each surrounding sequence gives it the prerequisites it needs. An inclusion that works in one context may depend on setup absent from another.

Do not assume that scanning the page source alone proves that an expanded inclusion ran. Inspect your tool's report and use a separate test of the included source or assembled content where needed. Likewise, a successful standalone test of an included file does not prove that the surrounding page introduces it at the right point. Make the relationship between source file, rendered page and test explicit in your coverage record.

mkdocstrings processes special expressions in Markdown to insert API documentation. An API page can therefore present examples authored in docstrings rather than directly in the page's Markdown. Docstring examples are code too: identify the source object, execute supported examples with their stated setup, and inspect the generated page as part of the build review. The Python docs testing guide goes deeper on imports, examples and Python environments.

A generated reference and a tutorial may use the same API in different ways. Test both when they make different promises. For the reference, check the smallest example and its expected result; for the tutorial, check the whole sequence a reader follows. The SDK documentation guide discusses connecting those checks to a release.

Run the page in a clean container

A clean starting environment helps reveal packages, files and settings that are present on a maintainer's machine but absent for a new reader. Use one fresh environment for each page or independent reader journey. Within a page, preserve the order of the steps and the state that a reader would carry forward. If the page explicitly depends on an earlier page, also test that cross-page journey; a fresh container for each page does not by itself establish that the full journey works.

Separate documented setup from test-only setup. If an example requires an installed package, decide whether the page installs it or names it as a prerequisite. Setup supplied by a harness can make a test runnable, but it can also hide an instruction that a reader needs. Record what the harness supplied, what the page supplied, and which artifact or environment was tested.

DocsAlive plans supported blocks in page order and uses a fresh container per page. Its containers run as an unprivileged user with a read-only root filesystem, dropped Linux capabilities, no privilege escalation, and resource and time limits. It can compare documented output shown after a block. A pass can represent execution or a syntax check, so inspect the status and treatment before describing a page as fully executed.

Use the CLI's repeatable --setup option for a shell command needed at the start of each page's container. Keep that command visible in the test record and do not mistake it for a documented install step. The CLI reference describes selection, setup and reporting options; how it works describes the stages of a run.

A generic CI outline

The following shell commands illustrate separate build and example checks. Run them from the repository root in an environment where the site dependencies and the CLI are already available. The first command's output is informational; it does not install the listed dependencies. Adapt the include pattern to the repository's actual docs directory.

mkdocs get-deps
mkdocs build --strict
docsalive scan . --execute --include 'docs/**' --out docs-test-report

The scan command tests supported extracted blocks, subject to its plan and skips. It does not render the site or expand every possible reader journey. Review docs-test-report/report.md and docs-test-report/report.json, including skips and hints, after the run. If the page requires an explicitly documented local package setup, add an appropriate --setup command and document that decision. Run against the release artifact or published package as a separate check when that is what the reader will install.

Choose a CI schedule that matches your changes: run relevant pages when documentation or an API changes, check the release path, and repeat periodically to catch dependency drift. The quickstart testing guide explains the reader journey; the CI guide has a platform-specific workflow if you need one later.

An open-source option for Python blocks

The mktestdocs README describes using pytest with check_md_file to run Markdown code blocks beginning with a Python fence. Its example parametrizes Markdown files under a documentation directory. Assertions in those blocks can also check expected behavior. This is useful when your main need is to run Python examples under a test suite you maintain.

The README says its standard settings assume blocks are independent; memory=True supports blocks that depend on earlier blocks. Select the setting that matches the page's instructions. If a page uses a variable defined earlier, an independent test can produce an artificial failure; if a page promises standalone snippets, shared state can conceal a missing definition.

Decide separately how to cover shell installation, tab alternatives, included source and generated API pages. The README's Python-block example does not by itself establish coverage of all of those paths. The free docs testing tools guide offers a starting point for choosing a combination of checks. Keep the resulting coverage record close to the test configuration so a future maintainer knows which reader paths remain untested.

Triage failures before changing the docs

An execution error is evidence to investigate, not automatically proof that a published instruction is wrong. Compare the actual error, the page's promises, the setup supplied by the test, and the relevant source location. DocsAlive classifies findings using the actual error rather than the exit code alone; its report includes the category and reason.

A timeout, runtime error or external network result can need human review. A different timestamp or ordering in output can also be harmless, while a changed value may contradict the page. Do not erase the evidence by updating expected output until you know what changed. A focused reproduction and a link to the source make review faster.

Checklist for an MkDocs example test

Use the broader docs testing checklist to assign owners and decide when each check runs. A checklist is useful only if a skipped item remains visible: a green build with an untested tab is still a partial result for that page.

What this does not replace

DocsAlive does not render your MkDocs site or check links. Continue to build and inspect the site, including navigation, formatting and links. Human review is still needed for clarity, accessibility and whether a page teaches the intended task. A code block can run and still be misleading in its surrounding explanation.

Go, Rust, Java and other compiled-language examples are not executed by the product. Full TypeScript and JSX are not judged; simple TypeScript annotations can be stripped for execution. Examples requiring credentials are classified rather than executed with real keys, and running them with customer-provided test credentials is not currently supported. Report these boundaries alongside results, and use separate appropriate checks for the paths that matter.

For a starting point, the free health check lets you send a public repository you own or maintain; the team runs the engine on up to 40 of its docs pages and emails the real report within 2 business days; the checkout is deleted after the run. Pricing is Open source $0/month, Team $299/month, Growth $699/month, and Scale from $1,499/month.

If your site uses Docusaurus or Mintlify instead, see testing Docusaurus documentation examples and testing Mintlify documentation examples.

If your project uses Sphinx, see testing Sphinx documentation examples.

The sources we checked

FAQ

Does mkdocs build --strict run my examples?

The command builds the documentation and aborts on warnings. It does not establish that the commands and code shown to readers execute successfully. Run a separate example check and retain both results.

What does mkdocs get-deps tell me?

It shows required PyPI packages inferred from plugins in mkdocs.yml. It is not a complete list of dependencies used by tutorials, included examples or reader environments.

Should I test every Material tab?

Test each reader path that matters, starting alternatives in equivalent clean environments. DocsAlive selects one variant per tab group and skips the others, so one run does not prove that every alternative works.

How do I test an included snippet?

Trace the inclusion to its source file and test the content readers see in the context of the page. Check the report to establish what your tool actually extracted; a raw page-source scan alone is not proof that an expanded inclusion was tested.

Can I test docstring examples?

Yes. Identify examples in the source docstrings used for generated API pages and test them with the appropriate setup. Also inspect the generated page, because running the source example does not establish how it appears in the built site.

Why keep blocks in page order?

A later block may depend on an earlier import, variable, installation or file. Preserving the reader's sequence avoids treating a valid dependency as a failure; a clean environment for each page helps reveal dependencies that the page never supplies.

Does a skipped example mean the page passed?

No. A skip records that a block was deliberately not run and gives a reason. Review the reason and arrange another check if the omitted path is important to readers.

Can the product check the finished site and its links?

No. It does not render the MkDocs site or check links. Keep the strict build and any separate site and link checks alongside example execution.