Testing Mintlify documentation examples

The short answer

Test Mintlify documentation examples as reader paths, alongside the checks for the documentation site itself. Use the Mintlify CLI to preview the site, validate it and investigate broken links; then separately run the supported code and commands in clean environments. A valid page can still tell a reader to install a missing package or call an API incorrectly. A working snippet does not prove that the page renders correctly or that its links work.

Inventory the MDX pages, fenced blocks, prerequisites and alternatives. For each CodeGroup or tab set, decide which choices you promise to readers and how each will be tested. Preserve the order of dependent steps on a page, start independent pages clean, and keep the selected variant, skipped blocks and actual errors in the report. Review both the page source and the rendered experience when presentation affects what a reader copies.

DocsAlive detects Mintlify from docs.json or mint.json, reads MDX, recognizes CodeGroup and Tabs as alternatives, and picks one variant per group. It plans supported examples in page order and runs them in a fresh sandbox container per page. It does not run the Mintlify CLI, render the site, check links or test API playground requests. The docs testing overview and how it works explain the wider workflow.

Find the configuration and MDX pages

The global settings reference describes docs.json as the configuration file. Start there and inspect the repository's actual page layout before choosing a test target. The configuration and the pages answer different questions: one describes site settings, while MDX page source contains the instructions and examples a reader encounters. The product also recognizes mint.json for detection; that detection rule is not a claim that both files have the same settings or roles.

List the pages in scope and the task each page promises. Include quickstarts, installation pages and API examples as well as short snippets embedded in explanations. Record paths and source lines for findings so a maintainer can locate the instruction. If a repository contains more than one documentation area, verify that the chosen target and include pattern reach the intended pages. A small, clean report from the wrong directory is not evidence about the rest of the site.

Compare the page inventory with the report's extracted blocks and reasons. An excluded page, an extracted block skipped for credentials, and a tested block that fails are three different outcomes. Assign each a next action. The static-site documentation testing guide explains why source, built site and published reader experience need distinct checks.

Think about the surrounding prose as part of the test. A Python block may rely on a package installed earlier, while a request example may assume an account, a token or a local file. The code may be syntactically sound and still be unusable because the reader was never told how to meet a prerequisite. Capture that prerequisite in the coverage record, including whether the test harness supplied it or the page actually documented it.

Use the Mintlify CLI for site checks

The CLI command reference describes mint dev as a local documentation preview. Use it to inspect page layout, tab labels, code formatting and the flow of the instructions. A preview gives a reviewer an opportunity to see the rendered page; looking at the page is still necessary. A server starting successfully does not itself establish that a reader's shell commands or API calls work.

Use mint broken-links to investigate links and mint validate to validate the build according to the CLI's checks. The installation guide states that validation exits with an error when a rule reports an error, while warnings do not fail the job. Retain and review warnings as well as the exit status. Check the command reference for the applicable options and interpret results in the context of the site being checked.

These checks are complementary. A broken-link result points toward navigation or references; it does not install an SDK or compare output from a tutorial. Validation can reveal site problems but does not prove that a fictional or stale package name exists. An example run can catch an import error while missing a bad link or a confusing tab label. Keep separate results so a reviewer can tell which signal supports a claim of readiness.

When a site check fails, inspect the diagnostic at the relevant page or configuration. When example execution fails, reproduce the reader's steps and examine the actual runtime error. A correction may require both: changing an example's source can also change the rendered page, while moving a page can affect links without changing the code. Recheck the affected evidence after the fix.

Understand code block presentation

The code formatting reference describes fenced blocks with a language followed by metadata. Supported presentation options include titles, syntax highlighting, line highlighting, line numbers and icons. Blocks are copyable, and a CodeGroup can present multiple blocks as tabs whose titles supply labels. These features make examples easier to navigate, but their presence is not proof that the commands work.

Treat the language label as an input to both presentation and the test plan. A highlighted language may be recognized by a renderer even if an example runner cannot execute it. A title may tell the reader where a file belongs or which variant to choose. Check that the title is accurate, the code can be copied as intended, and the surrounding prose explains what to do with it. A result tied only to the raw source should not be described as a review of the rendered copy experience.

Line highlighting can draw attention to a changed argument or an important line. After editing the block, check that the highlighted lines still support the explanation and that a reader copying the complete block gets a coherent example. Line numbers and visual emphasis do not supply missing imports, credentials or a file. If a snippet is intentionally partial, make that context clear instead of treating a standalone execution failure as automatic proof of a documentation bug.

Inspect interactive presentation separately from runtime correctness. A tab can display the expected code while its request fails, or the code can work while a label directs a reader to the wrong language. Keep evidence of both the page review and execution. The API reference guide provides a broader way to track examples tied to endpoints and changing behavior.

Treat CodeGroup and Tabs as alternatives

The CodeGroup reference describes grouping code blocks, and the Tabs reference describes tabbed content. A group can offer different languages or ways to complete the same task. The test must make an explicit choice: executing one tab is evidence about that variant, while running every tab one after another in a shared environment can create a journey no reader follows.

This illustrative MDX excerpt shows a bash request and a Python request to a fictional host. The package name acme-docs-demo is fictional too; the examples use standard request mechanisms rather than claiming that package exists. The environment variable is a placeholder, not a real key. The endpoint and response are illustrative and should be replaced with the real service contract in an actual page.

# Request the demo

Set `ACME_API_TOKEN` in your environment before choosing a request.

<CodeGroup>
```bash cURL
curl --fail --header "Authorization: Bearer $ACME_API_TOKEN" \
  https://api.acme-docs-demo.test/demo
```

```python Python
import os
from urllib.request import Request, urlopen

request = Request(
    "https://api.acme-docs-demo.test/demo",
    headers={"Authorization": "Bearer " + os.environ["ACME_API_TOKEN"]},
)
with urlopen(request) as response:
    print(response.read().decode("utf-8"))
```
</CodeGroup>

The two variants express a similar request, but they have different prerequisites and failure modes. The shell path needs the request tool and an environment variable; the Python path needs its runtime and the same variable. Test each supported language you promise with an appropriate authorized environment, and compare the behavior with the actual endpoint contract. Do not place a real credential in the page, test report or repository.

DocsAlive recognizes CodeGroup and Tabs as alternative groups, chooses one variant, and records other variants as alt-variant skips. A passing selection does not establish that the other language works. Credential-dependent examples are classified rather than executed with real keys, and customer-provided test credentials are not currently supported. The fictional request above therefore illustrates a coverage decision, not an executable product verification. Arrange a separate authorized check if the request is important to readers.

Review the path beyond the group too. If the next block assumes a variable, file or output produced only by one language, another tab may leave the reader stranded. Define equivalent starting conditions and follow each choice to the next shared step. If the alternatives genuinely require different preparation, say so beside the choice and retain that preparation in the test record. The installation guide applies the same reasoning to package-manager choices.

Run page steps in a clean environment

A useful example test preserves the page's sequence. A later snippet can depend on an earlier import, environment setup, installation or file. Starting every block in isolation can create artificial failures; running unrelated pages in one persistent environment can hide missing prerequisites. Use a fresh environment for each independent page, preserve state in page order, and separately exercise a journey that intentionally spans multiple pages.

DocsAlive plans supported blocks in page order and starts a fresh container per page. The container uses an unprivileged user, a read-only root filesystem, dropped Linux capabilities, no privilege escalation, and resource and time limits. It can execute, syntax-check or deliberately skip a block with a reason. An output block immediately following code can be used for comparison. A pass can reflect execution or a syntax check, so inspect the plan and report before claiming that every example ran.

Python and JavaScript examples can be executed, as can supported shell and curl examples. Simple TypeScript annotations can be stripped for execution, but full TypeScript is not judged. JSON and TOML can be validated. A language label by itself is not a guarantee of execution: Go, Rust, Java, Ruby and PHP are recognized but not executed. JSX and TSX are not judged yet. Plan separate coverage where a skipped language carries an important reader promise.

Use repeatable --setup commands only with a clear record of what they supply at the start of each page container. A harness-installed dependency may be appropriate for a page that explicitly states a prerequisite; it can also hide a missing install instruction. Record the artifact under test, network assumptions, fixtures and setup. The CLI reference explains the available selection and reporting options, and the Python documentation testing guide discusses Python-specific examples.

When a documented response includes a timestamp, identifier or other changing field, decide which parts are stable promises. Compare meaningful behavior without blindly replacing expected output whenever a test differs. An output difference can be harmless, but a changed field can also reveal a real mismatch with the prose. Keep the actual result and reasoning available to the reviewer.

A generic CI outline

From the documentation directory, with both CLIs already available, these shell commands illustrate distinct checks. Adapt the scan target and include pattern to your repository; the pattern shown selects MDX files below the current directory. Run the preview separately when a reviewer can inspect it, since it is a local server rather than a finite example test.

mint broken-links
mint validate
docsalive scan . --execute --include '**/*.mdx' --out docs-test-report

Review the CLI output, including validation warnings, and inspect docs-test-report/report.md and docs-test-report/report.json for statuses, categories, reasons and skips. The scan runs supported extracted blocks according to its plan; it does not invoke the Mintlify commands. If setup is needed, add a suitable --setup command and note whether the page supplies equivalent instructions. Use mint dev for a local visual review and retain the result as a separate check.

Run the appropriate checks when documentation and APIs change, against a release artifact when that is what readers install, and periodically when external dependencies can drift. A test of a local checkout is useful evidence about that checkout; it is not automatically evidence that a published package or external endpoint behaves the same way. The quickstart testing guide helps map the complete reader journey.

Triage failures before editing

A failed command is evidence to investigate, not automatically a docs defect. Compare the actual error with the documented steps, the intended artifact, the setup supplied to the test and any earlier failure. DocsAlive classifies findings using the actual error rather than the exit code alone, and reports a category and reason. That context helps a maintainer choose whether to fix a page, adjust a test or record a gap.

Timeouts, runtime errors and some external network results need human review. A network failure may be transient or may reveal a broken instruction; investigate before changing a page. A placeholder package is illustrative rather than proof of a broken distribution. For a real package, reproduce the failure against the intended artifact, make the smallest correction that addresses the reader's problem, and rerun the relevant page and affected variants.

Keep the source location and enough error evidence for another maintainer to understand the decision. Do not remove an example merely to make a check green without considering the reader's task. If a check cannot cover a path, document the gap and its owner. The JavaScript documentation testing guide covers runtime-specific questions, while the README testing guide helps with examples outside the site pages.

Checklist for a Mintlify example test

Use the broader docs testing checklist to assign owners and schedules. Revisit the inventory when a group gains a language, a page adds a prerequisite or an API response changes. The report should make it possible to distinguish tested paths from paths still awaiting a suitable check.

What this does not replace

DocsAlive does not run the Mintlify CLI, render the site, check links or test API playground requests. Continue to use the site checks and inspect the reader experience. Human review is needed for clarity, accessibility and whether the explanation makes sense around a working snippet. A successful code block does not establish that a playground request works or that a reader can discover the right tab.

Go, Rust, Java, Ruby and PHP are recognized but not executed; other recognized languages also have this limitation. JSX and TSX are not judged yet, and full TypeScript is not judged, although simple type annotations can be stripped for execution. Credential examples are classified rather than executed with real keys; customer-provided test credentials are not currently supported. Use separate appropriate checks for the paths that matter and retain those limitations with the results.

The free health check lets you send a public repository you own or maintain; the team runs the engine on up to 40 of its docs pages and emails the real report within 2 business days; the checkout is deleted after the run. Pricing is Open source $0/month, Team $299/month, Growth $699/month, and Scale from $1,499/month.

The sources we checked

FAQ

Does mint validate execute documentation examples?

Validation checks the build according to its rules. It does not establish that every command or API call shown to readers works. Run an example check separately and review warnings as well as errors.

What is mint dev useful for?

It starts a local preview. Inspect the rendered page, including labels, formatting and the reader's sequence. Starting a preview alone does not verify that a copied command succeeds.

Does mint broken-links test API requests?

It is a link check, not a test of the behavior promised by a request example or an API playground. Use an appropriate separate request test and keep its credentials and results under control.

Should every CodeGroup language be tested?

Test every variant you promise using an appropriate environment. DocsAlive picks one variant per group and marks the others alt-variant, so a single selection does not cover every language.

What if a highlighted language is not executed?

Presentation and execution are separate capabilities. Review the report for the block's status and arrange a suitable language-specific check when the runner does not execute that example.

Why run blocks in page order?

A later block may depend on a previous installation, variable or file. Preserving order reflects the reader's sequence; a fresh environment for each independent page helps reveal undocumented dependencies.

Can credential examples run with my test keys?

The product classifies credential-dependent examples rather than executing them with real keys. Running them with customer-provided test credentials is not currently supported. Arrange a separate authorized check for the request.

Can the product replace site and playground review?

No. It does not run the Mintlify CLI, render the site, check links or test playground requests. For a different MDX framework, see the Docusaurus testing guide.