On this page
The short answerWhat belongs in a documentation pipelineHow a GitLab job runs a checkTwo ways to invoke the checkSet up the reader's environment deliberatelyUnderstand what the scan reportsDecide when to run and what to reviewDesign the job around a reader promiseTriage failures before changing the job or pageChecklist for a GitLab documentation jobWhat this does not replaceThe sources we checkedFAQThe short answer
Put a documentation example check in a GitLab CI job alongside the documentation build and the project's other tests. A job can start from a suitable image, install the dependencies a reader needs, and run a repository script or docsalive scan after the CLI has been set up during onboarding. Make a failing check fail the job, and review the evidence before deciding whether the documentation is wrong.
A successful pipeline that only builds the site or installs packages does not prove that examples ran. Run checks for merge-request changes and on a schedule so both reviewed changes and later dependency drift have a chance to surface. The GitHub Actions guide covers the same distinction in a sibling CI workflow.
DocsAlive does not ship a packaged GitLab CI component. Its CLI and GitHub App are set up during onboarding, and a public Git URL, including a public GitLab repository, can be scanned. The product executes supported Markdown examples in sandbox containers and triages findings; compiled languages are not executed, and TypeScript support is limited. Keep native language tests and site checks alongside it.
What belongs in a documentation pipeline
Documentation changes can break a reader's path even when application tests pass. An install command can name the wrong package, an import can refer to an API that changed, or shown output can lag behind behavior. A page can also depend on a file, tool or credential that its instructions never introduce. These are different failures and deserve different responses.
A merge request is a useful point to test a change while the author and reviewer can still relate the result to the diff. A schedule adds another signal after the change has landed: a dependency or external condition may change even when no one edits the page. The GitLab pipeline reference describes merge-request and scheduled pipelines. Choose the cadence and scope that fit the repository, and make sure someone reviews scheduled failures.
Separate the signals. A documentation build checks whether the build completes under its configuration. An installation step establishes that a particular installation command succeeded in its environment. Neither, on its own, establishes that later examples ran or that their results matched the page. An example test should execute the relevant reader steps and check meaningful outcomes when possible. Review of prose, navigation and presentation remains a separate task.
Begin with an inventory of high-value reader paths: installation, a first successful use, common examples and the README. The installation guide and README guide help identify those entry points. Record the page, the dependencies it assumes and the command that provides coverage. A small, explicit inventory is more useful than a green job whose scope nobody can explain.
How a GitLab job runs a check
GitLab defines pipeline jobs in .gitlab-ci.yml. The jobs reference describes jobs as the fundamental elements of a pipeline: they have commands for tasks such as building or testing, execute on a runner, and have a job log. A job is defined at the top level with a unique name and a script or a trigger. The script keyword supplies commands for the job to execute.
The YAML reference describes image as the container image used for a job, and a job can override a default image. Choose an image with a suitable starting environment for the check. Then make the dependencies and setup explicit, either in the repository script or in commands that prepare the environment. The fact that an image contains a tool should not hide a prerequisite from readers if the page expects them to install it.
A job is a useful boundary for a focused result. Name the job for the behavior it tests, keep its log understandable, and preserve enough context to reproduce the failure locally. A clean starting environment helps expose assumptions carried on a maintainer's machine. Within a reader journey, retain the order of steps that build on one another; between independent alternatives, start from equivalent clean conditions.
Two ways to invoke the check
The following illustrative .gitlab-ci.yml defines two alternative approaches. Choose the job or jobs appropriate to your repository. The image is a placeholder for an image available to your runner and prepared with the dependencies and tools the selected job needs; replace it with your actual image. The repository script is also illustrative and must exist and return a failing status when its check fails. The second job assumes the product CLI has already been set up during onboarding and is available in its environment.
docs_example_script:
image: acme-docs-demo
script:
- sh scripts/check-doc-examples.sh
docs_example_scan:
image: acme-docs-demo
script:
- docsalive scan . --execute --include "docs/quickstart*"
The script can install reader dependencies and run the project's chosen tests. Make its setup and failure behavior reviewable in the repository. If it uses a native test for a compiled language, retain the output that identifies which documentation examples were covered. Test an intentional failure when setting up the job so the team can verify that a failed check makes the job fail.
The scan command selects matching pages and requests execution. Without --execute, the CLI extracts and plans rather than running the plan. Adapt the include pattern to the actual repository and inspect the report for each block's treatment. The CLI reference describes the local target, include patterns, setup and reporting. A successful command is useful only when the selected pages and skipped cases are understood.
The example does not define when GitLab creates a pipeline or how a team configures its schedule. Set those choices in the project's actual CI configuration and settings, using the pipeline reference for the relevant pipeline types. Confirm in a real merge request and a scheduled run that the intended job appears and reaches the intended pages. The GitHub Actions guide offers a comparison of the same coverage decisions on another platform.
Set up the reader's environment deliberately
A documentation check should start with the assumptions a reader is asked to satisfy. List the runtime, package, installer and local files needed by the example. Install what the page tells the reader to install, and distinguish that from infrastructure supplied only to make the job possible. If the check provides a package the page never mentions, a green result may conceal a missing instruction.
Use a clean image or environment for independent paths. A long-lived environment can contain a tool or file left by another task and make an incomplete tutorial appear to work. At the same time, preserve state within a page when later blocks depend on earlier imports, variables or installation steps. A test that resets after every block can produce a false failure for a valid sequence; a test that shares everything across unrelated pages can hide a missing prerequisite.
When a page presents package-manager alternatives, test each important path independently. Record the one tested and the environment used. If the page includes snippets from other files or generates reference content, trace the reader-visible example to its authoring source and establish a check that actually reaches it. The static-site guide explains why the source page, built site and tested content can differ.
Avoid making a test dependent on production credentials. DocsAlive classifies examples that need credentials rather than executing them with real keys, and it does not currently support running them with customer-provided test credentials. Keep that boundary visible in the coverage record, and arrange a separate authorized check where a credential-dependent reader journey matters.
Understand what the scan reports
The product reads .md, .mdx and .markdown pages under the selected docs directory, extracts fenced blocks and plans how to treat each one. It runs supported examples in a fresh sandbox container per page and keeps blocks in page order. The container uses an unprivileged user, a read-only root filesystem, dropped Linux capabilities, no privilege escalation, and resource and time limits. The how it works page describes the stages in more detail.
A pass can mean that a block was executed or syntax-checked, with shown output matched where applicable. A warning can reflect differing output or a dependency warning. A failure has a category and a reason; a skip identifies a deliberate omission. Read those details before saying that an entire page ran. The report can distinguish an actual likely documentation bug from context, missing fixtures or other conditions that require a different response.
Use --setup when each page's container needs a shell setup command, and document what that setup supplied. Use --out to write Markdown and JSON reports when a durable local report is useful. The JSON findings identify the page, line, language, status, category and reason. A line-level source location helps a reviewer find the actual instruction to fix. Keep a separate record for content or paths the scan did not reach.
Python, JavaScript and shell examples are among the supported executable fences; curl and HTTP examples are sent with curl, and JSON and TOML are validated. Simple TypeScript annotations can be stripped before execution, but other TypeScript-only syntax is not judged. JSX and TSX are not judged. Go, Rust, Java, Ruby and PHP are recognized but not executed. The frameworks reference describes the supported scope, while the Rust guide explains why a native Rust check must remain in the pipeline.
Decide when to run and what to review
A focused merge-request check gives reviewers feedback while a change is being discussed. Include affected pages and any shared setup or API changes that could affect them. A broader scheduled run can discover drift outside the changed files. Keep a clear route from a scheduled result to an owner; unattended failures quickly stop being useful evidence.
A release can change the package a reader installs, so compare the test's package and environment with the artifact the page describes. A check of local source and a check of the distributed package can answer different questions. Decide which promise each job makes, and avoid claiming release coverage from a job that only tested a different artifact. If a tutorial crosses pages, add a separate end-to-end check for that journey rather than assuming individual page containers establish the whole sequence.
Review the actual report, including skipped blocks. A green result with an unsupported language or a credential-dependent step is partial coverage, not evidence about the omitted operation. A build and an example check can both pass while a page is unclear or a link is wrong. Assign the remaining reviews explicitly, using the docs testing checklist as a starting point.
Design the job around a reader promise
Start with one complete path rather than a collection of unrelated commands. For an installation journey, the promise might be that a reader can install the package, import it and obtain a particular result. Write down the prerequisites, the package source, the commands in order and the expected result. Then make the repository script exercise that path in a clean environment. An install command that succeeds is one step in the evidence; it does not establish that the next import or operation succeeds.
A separate job can cover a different reader entry point or language. Keep a native language check where the scan does not execute the language, and identify which documentation source it reaches. If a README is part of the reader journey, verify that the selected test actually includes it. The README guide and Rust guide help with these coverage decisions. A page outside the selected scope needs its own check or an explicit record that it remains uncovered.
Make a failure actionable
A useful job log identifies the first failing step, the page or source file, the command and enough environment context for a maintainer to reproduce the problem. Preserve the distinction between setup and the example itself. If setup fails before the example starts, a maintainer should not have to guess whether the page's code ever ran. Likewise, if the example runs but produces different output, retain both the expected and actual result when available.
Review variable output with care. A timestamp, ordering difference or external response can differ without proving that an instruction is wrong. A changed value central to the lesson can be a real documentation problem. Compare the failure with the promise in the page and the environment the test supplied. Once the cause is understood, fix the appropriate source or test setup and rerun the relevant job. Link that verification to the change so reviewers can assess what was repaired.
Keep scope and schedule visible
For each run, retain the selected directories or patterns and the test command. A narrow merge-request check can give fast feedback, but its success should not be described as coverage of every page. A periodic run can cover a broader inventory and catch changes in dependencies or external conditions. Review both the failure and the list of skips, because an unchanged skipped example can leave a significant reader path unverified for a long time.
When the team changes its docs framework or moves pages, compare the new source layout with the job's include patterns. A job can keep succeeding after it stops selecting the intended material. Confirm the selected files in the actual report, update the inventory and rerun the check. The static-site guide and GitHub Actions guide provide more context for relating source selection to CI evidence.
A test that depends on an included file should identify both the page and the authoring source. If the same content appears in several places, a source-level check can help, but the surrounding pages still need the correct prerequisites and explanation. Treat the built page, the underlying source and the execution result as complementary evidence. This gives the person receiving a scheduled failure a clear route from the symptom to the appropriate fix.
Triage failures before changing the job or page
| Finding | What to investigate | Next action |
|---|---|---|
| Site build fails | Build configuration, source and log | Fix and rerun the build separately |
api-drift |
A supported snippet refers to an API that changed | Reproduce with the intended package and correct the instruction |
invalid-syntax |
A complete snippet does not parse | Fix the code or make the intended fragment clear |
stale-output |
The code runs but the documented output is old | Confirm the expected result before updating it |
package-not-found |
A documented package name is unavailable | Verify the name and distribution path |
missing-install-step |
A page uses a package it does not install | Review the hint and the reader-facing prerequisites |
missing-fixture or depends-on-context |
The test lacks a file or surrounding state | Compare the test setup with the page's instructions |
needs-credentials |
A block requires credentials | Record the gap and arrange separate authorized coverage |
unsupported-lang |
A recognized language was skipped | Run the appropriate native test |
runtime-error, timeout or external network result |
A failure needs review | Preserve the error and reproduce before assigning a cause |
Triage uses the actual error rather than an exit code alone. Do not automatically rewrite documentation because a command failed in CI: an unavailable external service or a missing test fixture can produce a failure without proving the page is wrong. Conversely, a missing install instruction may matter to a reader even when a harness can supply the dependency. Capture the decision and rerun the affected check after a fix.
Checklist for a GitLab documentation job
| Check | Evidence to retain |
|---|---|
| Identify reader journeys and source pages | Scope, owners and relevant file locations |
| Configure a job with a suitable image and script | Job definition and dependencies supplied |
| Verify failure propagation | A deliberate failed check produces a failed job |
| Run on changes and periodically | Observed merge-request and scheduled results |
| Keep build and example signals distinct | Separate command results and logs |
| Test alternatives independently | Selected path, clean starting state and result |
| Inspect the scan report | Per-block treatment, reasons and relevant evidence |
| Cover native language and credential gaps | Separate check or recorded limitation |
| Triage, fix and rerun | Cause, changed source and verification |
The GitHub Actions guide discusses the same evidence and ownership questions for a sibling CI platform. Use the free tools guide when deciding how native tests and other checks fit together.
What this does not replace
DocsAlive does not provide a packaged GitLab CI component. The CLI and GitHub App are set up during onboarding; source options include a local checkout and a public Git URL, including a public GitLab repository. Do not infer that a GitLab App, a downloadable component or a ready-made pipeline is supplied. Configure the job and the environment for your repository.
The product does not render the docs site or check links. It does not execute compiled languages, including Rust and Go, and it does not judge full TypeScript or JSX/TSX. Credential examples are classified rather than run with real keys. Keep site builds, native tests, link checks and human review in the pipeline and review process. The VitePress guide shows how this separation applies to a site framework, and the docs testing overview explains the broader approach.
For a starting point, the free health check accepts a public repository you own or maintain; the team runs the engine on up to 40 of its docs pages and emails the real report within 2 business days; the checkout is deleted after the run. Pricing is Open source $0/month, Team $299/month, Growth $699/month, and Scale from $1,499/month.
Browse the guides or contact the team for questions about the workflow.
The sources we checked
| Source | What it covers |
|---|---|
| GitLab CI YAML reference | Pipeline configuration, image and script keywords |
| GitLab jobs | Job definitions, commands, runners and logs |
| GitLab pipelines | Pipelines, including merge-request and scheduled runs |
FAQ
Does a successful docs build prove examples work?
No. A build establishes that the site built under its configuration. Run the examples separately and check the behavior the page promises.
What does a GitLab CI job contain?
A job is a pipeline element defined in .gitlab-ci.yml. Its script can specify commands to execute, and the job log records execution.
Can a job choose its own image?
Yes. A job can specify an image and override a default image. Make sure the chosen environment has the tools and dependencies the check needs.
Should the check run on a schedule?
A scheduled run can catch drift after a change is merged. Assign someone to review the result and retain enough context to reproduce a failure.
Is there a packaged product component for GitLab CI?
No. The product does not ship one. Set up the CLI during onboarding and configure your own job to invoke the check.
Can a public GitLab repository be scanned?
Yes. A public Git URL is a supported source, including a public GitLab repository. A local checkout is another source option.
Does a passing scan cover Rust and Go examples?
No. Compiled languages are recognized but not executed. Use native tests and retain their results alongside the scan.
What if the report skips a credential example?
Treat it as a coverage gap, not a pass for the operation. Arrange a separate authorized check if that reader path is important.
