Testing Rust documentation examples

The short answer

Use Rust's own documentation tests to check Rust examples. cargo test runs documentation tests by default along with other tests, and cargo test --doc focuses on documentation tests. A regular rustdoc test passes when it compiles and runs without panicking. Put assertions in examples when the result matters, and keep examples that are meant only to compile distinct from examples that are meant to run.

DocsAlive recognizes Rust fences but does not execute them: they are skipped as unsupported-lang. It does not run the Rust compiler, Cargo or rustdoc tests. A passing DocsAlive report is not evidence that Rust examples compiled or ran. Use it for supported Markdown examples on the same site, and retain native Rust checks for the crate and its documentation. The Go guide explains a similar compiled-language limitation.

Why Rust documentation examples drift

A documentation example is a promise about an API in a particular context. A function can be renamed, its arguments can change, or a return value can require different handling. An example that was once correct may no longer compile against the crate readers install. Even when it compiles, an assertion can expose a behavior change that compilation alone cannot detect.

Edition differences matter too. An example may be interpreted under an edition different from the one its author assumed. Record the intended context and use the rustdoc edition attributes when appropriate. Treat an edition choice as part of the test's meaning, not as a decorative label. When a failure appears after an API or edition change, reproduce it with the actual crate and test configuration before changing the prose.

A sample can also compile without ever being run. That may be intentional for a potentially disruptive or environment-dependent operation, but it leaves a different kind of evidence from a normal passing doctest. The distinction matters when a page promises a computed value, an error, or a sequence of effects. A build of the documentation site is useful in its own right, but it cannot establish that a Rust snippet was executed.

Make an inventory of the examples readers encounter: crate documentation, module documentation, README content included in the tested documentation, and separate Markdown pages on the site. Note which test actually reaches each source. A native test for a library's documentation comments does not automatically establish coverage of every separate Markdown page. The README guide and docs testing checklist help make that inventory reviewable.

What rustdoc tests do

rustdoc documentation tests extract examples from documentation and run them as tests. In a regular doctest, successful compilation and execution without a panic produce a pass. An assert_eq! can turn the value promised by a page into a check: if the value changes unexpectedly, the assertion panics and the test fails. A successful run is evidence about the tested example and its environment, not every possible input to the API.

A fenced block without a language is assumed to be Rust by rustdoc. An explicit rust label can make the intent clear when Markdown is also consumed by other tools. The Cargo test reference says cargo test compiles and executes unit, integration and documentation tests; documentation tests run by default and rustdoc extracts samples from library-target documentation comments. cargo test --doc selects the documentation tests. Check your actual test output and configuration to establish which examples were selected.

A README or module page needs the same attention as the main crate page. The Rust book's documentation discussion describes testing documentation comments and including a README in crate documentation. Where a README is included in documentation rustdoc tests, keep it in the native test path. For a separate Markdown page, establish a test path explicitly rather than assuming that a filename alone makes it a doctest.

What rustdoc adds before testing

Rustdoc preprocesses examples to make short samples usable as tests. It inserts common allow attributes, including unused_variables, unused_assignments, unused_mut, unused_attributes and dead_code. It adds attributes configured for documentation tests and leaves leading crate attributes intact. If an example has no extern crate and the relevant crate-injection setting has not disabled it, rustdoc inserts an extern crate for the crate. If the example has no fn main, rustdoc wraps it in one. The rustdoc reference describes the full preprocessing rules.

This convenience is useful, but it affects how you interpret a test. The source displayed to a reader may be shorter than the program rustdoc compiles. If a reader needs a complete program outside that context, show the necessary surrounding structure or explain where it comes from. Do not infer that the displayed fragment is independently runnable in every setting just because its doctest passes.

A small crate example

Suppose the fictional crate acme_docs_demo exposes a function that adds one. A separate Markdown page could show a reader this complete use of the public function:

```rust
use acme_docs_demo::add_one;

let result = add_one(2);
assert_eq!(result, 3);
```

In a crate source file, the function and its documentation test could look like this:

/// Adds one to a number.
///
/// ```
/// use acme_docs_demo::add_one;
/// assert_eq!(add_one(2), 3);
/// ```
pub fn add_one(value: i32) -> i32 {
    value + 1
}

The example is fictional; provide the crate and its test environment in a real project. The assertion makes the expected value explicit. A printed line is not automatically checked merely because prose describes it: use a Rust assertion or another suitable explicit test for the behavior you promise. The separate Markdown fence also needs its own coverage plan if it is not included in the documentation rustdoc tests.

Hide setup without hiding the contract

Rustdoc lets an author hide lines starting with a single # from the displayed example while retaining them for the test. This is useful for setup or assertions that would distract from the concept being taught. For example, the following documentation comment tests the value while showing the use and result assignment to the reader:

/// ```
/// use acme_docs_demo::add_one;
/// let result = add_one(2);
/// # assert_eq!(result, 3);
/// ```
pub fn illustrated() {}

Hidden lines still influence the test. Review them as carefully as visible lines: a hidden import or setup step can make the test pass even though the page fails to tell a reader what they need. Keep the public promise visible in prose or code, and use hidden setup for details that are truly supplied by the surrounding context. The rustdoc reference also describes escaping a leading hash when it should appear in displayed code.

Choose attributes that match the promise

Rustdoc attributes follow the opening fence and determine how a test is interpreted. Choose one based on what the reader should expect, and review it whenever the surrounding example changes.

A negative example marked compile_fail is useful when the documentation explains code that should be rejected. It is not evidence that a valid reader journey runs. Similarly, should_panic checks for a panic rather than a normal return. An ignored example or a compile-only example needs a separate plan if runtime behavior is central to the page. Avoid making a test green by changing its meaning without also changing the promise to the reader.

Inspect warnings as well as results

Rustdoc documents rustdoc --test --test-args=--show-output and cargo test --doc -- --show-output as ways to show warnings in doctests. Unused warnings remain hidden by default; the reference describes enabling them for an example or at the crate root. Showing warnings is useful during maintenance, but the output setting does not turn a compile-only or ignored example into an executed one. Read the attributes and test results together.

A warning may explain a future maintenance problem even when the test passes. Record the actual warning, the affected source and the test environment, then determine whether the example or its surrounding explanation needs a change. A clean-looking output is only as broad as the warnings enabled for that run.

Run the checks in CI

Run the native checks whenever the crate or documentation changes and periodically to notice drift. The following generic shell outline assumes the Rust tools and the product CLI have already been made available during setup. Adapt the selected directory to your repository:

cargo test
cargo test --doc -- --show-output
docsalive scan . --execute --include "docs/quickstart*" --out docs-test-report

The second command makes documentation test output visible; it is not a replacement for inspecting the first command's broader result. The scan covers supported extracted Markdown fences, subject to its plan and skips. It does not execute Rust. Inspect the Markdown and JSON reports to see what was executed, checked or skipped. The CLI reference explains selection and setup; how it works describes the report and triage.

Use a clean environment and record what the test supplied. If setup installs a package or provides a file that readers do not have, decide whether the page should document that prerequisite. Keep steps that depend on each other in reader order, but test independent entry points from a fresh starting state. For a hosted pipeline, the GitLab CI guide and GitHub Actions guide discuss separate job-level checks.

Define the evidence for each example

Before wiring commands into CI, classify the promise made by each example. A positive example may promise that a public function accepts a value and returns a particular result. A negative example may teach that a particular piece of code is rejected. Another may show code that should panic. These are all useful documentation, but they need different expected outcomes. An inventory that records only “tested” loses that distinction.

For a positive example, identify the input, the operation and the observable result. An assertion can check the result directly, while the prose explains why it matters. If the sample has several independent claims, consider whether each one is visible in the test. A test that calls a function without inspecting the result can pass even when its returned value no longer matches the explanation. Conversely, an assertion about an incidental detail can create maintenance work without protecting a reader-facing promise.

For a negative example, preserve the reason the page presents it. A compile_fail test confirms the expected compilation outcome, but a maintainer should still read the explanation to ensure that it describes the intended lesson. A should_panic example has a different expected runtime outcome. When editing either example, review the attribute, the code and the text as a unit. If one changes without the others, a passing result can still leave the page misleading.

Review coverage when the crate changes

When changing a public function, search for its examples in crate documentation, module documentation and the separate site. Update each reader-facing occurrence or explain why a particular page intentionally shows a different context. Run the native tests after the edit, and inspect the selected tests rather than relying solely on the final exit status. A command can succeed while an ignored or compile-only sample leaves an important behavior untested.

A README included in tested crate documentation can provide valuable coverage at a common entry point. Check the inclusion and the test result in the actual project; a separate README or site page may need its own arrangement. A reader may arrive directly at a page without having followed the earlier tutorial, so verify that its stated prerequisites match the way it is intended to be used. The installation guide helps identify the setup that belongs in a reader journey.

Keep a record for examples that cannot safely or reasonably run in the normal test environment. State the reason, the behavior that remains unchecked and the owner of any separate verification. This makes an intentional exception visible to future maintainers. Revisit it when the API or environment changes instead of allowing a skip to become an unexplained permanent assumption.

Compare a native result with a site scan

The native Rust test and the Markdown scan answer different questions. A Rust doctest can compile and run a documented crate example, including code hidden from the displayed sample. The scan can exercise supported fenced languages on the same site and report a Rust fence as unsupported. Neither result should be used as shorthand for the other. Link the results in a change review so that a reviewer can see both the Rust evidence and the remaining site coverage.

For example, an installation page might contain a shell command followed by a Rust snippet. A supported shell check can reveal whether its documented setup works in the test environment. The Rust snippet still needs its own native check. If those checks use different package sources, record that difference: a passing local crate test does not automatically prove that the package a reader installs contains the same API. Review the Go guide for the same separation of a supported surrounding page and a compiled-language example.

After a failure, reproduce the smallest relevant case with the same crate and edition, note what the reader was told to do, and inspect both visible and hidden code. Change the source that owns the incorrect instruction, rerun the native test, and review the rendered page if the displayed explanation changed. Retain the evidence with the change so the next maintainer understands what the test protects.

Triage failures before editing

A raw failure is a starting point for investigation. A test environment can lack a fixture, use a different package, or supply context the page does not. Preserve the first error and the relevant source location before making changes. Then rerun the affected native test and any supported page scan after the fix.

Checklist for Rust documentation tests

What this does not replace

DocsAlive does not run rustc, cargo, cargo test, cargo test --doc or rustdoc --test. Rust examples are recognized but not executed and receive unsupported-lang; a passing report cannot prove they compiled or ran. It also does not render the documentation site or check links. Keep the site build, native Rust tests and human review of the explanation alongside its scan.

For supported Markdown fences, the product runs Python, JavaScript, shell and other supported examples or validates supported data formats. TypeScript support is limited to stripping simple annotations before execution; other TypeScript-only syntax is not judged, and JSX and TSX are not judged. Credential examples are classified rather than run with real keys. The frameworks reference and documentation testing overview describe scope; the Go guide covers the comparable compiled-language boundary.

For a starting point, the free health check accepts a public repository you own or maintain; the team runs the engine on up to 40 of its docs pages and emails the real report within 2 business days; the checkout is deleted after the run. Pricing is Open source $0/month, Team $299/month, Growth $699/month, and Scale from $1,499/month.

For a site using VitePress, see testing VitePress examples. Browse the other guides for language and workflow coverage.

The sources we checked

FAQ

Does cargo test run documentation tests?

Documentation tests run by default alongside other tests. Inspect the actual output and configuration to establish the examples covered by your run.

What does cargo test --doc check?

It selects documentation tests. A regular doctest must compile and run without panicking; its attributes may instead specify a different outcome or omit execution.

Does a passing Rust fence in the product report prove compilation?

No. Rust fences are recognized and skipped as unsupported-lang. Use rustdoc or Cargo for that evidence, as the Go guide recommends for its own compiled-language boundary.

Why hide a line with #?

A single leading hash can keep setup or an assertion in the test while hiding it from the displayed example. Ensure hidden setup does not conceal a prerequisite readers need.

Does no_run check runtime behavior?

No. It compiles the example but does not run it. Plan a separate execution check when the page promises runtime behavior.

How do I check a value returned by an example?

Use an assertion in a regular doctest. A failing assertion panics and fails the test, making the expected result part of the check.

Do warning display options show every unused warning?

No. Rustdoc says unused warnings remain hidden by default even when output is shown. Its documentation describes how to enable them.

Should separate Markdown pages be tested too?

Yes, if readers rely on them. Identify whether they are included in documentation rustdoc tests; otherwise provide separate coverage instead of assuming the native test reaches them.