Testing PHP documentation examples

The short answer

Test PHP documentation examples in layers: classify each block, lint copyable PHP, run complete examples in a clean directory, and compare stable results with assertions enabled. Check Composer instructions in a scratch project using the package and PHP versions the page names. Keep fragments, configuration, and examples requiring services visible as separate coverage decisions.

A syntax check is fast and useful, but it does not prove an example can load its dependencies or produce the documented result. A complete check follows the reader's setup, including the opening PHP tag, autoloading, namespaces, configuration, and any files created by earlier steps.

DocsAlive recognizes PHP fences but does not lint or run them. It reports them as skipped with not executable (php). Use PHP and, where appropriate, PHPUnit in your own CI; the product can still check supported blocks elsewhere on the page. See documentation code testing and README testing for related workflows.

Why PHP documentation examples break

A PHP example can look plausible in a browser while depending on details that disappear when a reader copies it into a new file. A missing <?php opening tag changes how a file is interpreted. A missing require 'vendor/autoload.php' leaves Composer-installed classes unavailable. A namespace or use statement may have existed in the author's source file but not in the published snippet.

Version drift can affect both code and installation. A function may have changed or been removed between PHP releases. A Composer package name or constraint may no longer represent the package the page intends readers to install. A test against a maintainer's existing vendor directory can hide a mistake in the page's installation instructions.

Runtime configuration is another dependency. An example might rely on a particular php.ini setting or an enabled capability without saying so. A local run that happens to inherit the right setting does not establish that a new reader can reproduce the result. State configuration requirements and test them deliberately.

These problems affect different documentation surfaces. A README usually needs a short path from installation to a first result. SDK documentation may assume a Composer project and a particular version of the library. An API reference may intentionally show just a method call. Each is useful, but the test must know which setup and surrounding code the reader is expected to have.

Finally, a successful process exit is not the same as correct output. If the page promises a value, verify that value. If it shows a representative response with fields that can vary, identify and check the stable parts.

Decide what each block represents

Before extracting PHP fences, record whether each block is meant to stand alone. This prevents a valid fragment from being judged as a complete file and prevents a complete example from passing only because the test silently supplies missing context.

For a fragment, document the enclosing function, class, namespace, or prior setup that makes it meaningful. Prepending an opening tag can make a fragment parse as PHP, but it does not prove that variables, imports, or dependencies are present. Similarly, a file can pass lint while containing code that fails only when executed.

When a page offers several alternatives, test each supported path independently. Do not combine a Composer setup from one tab with source from another unless the page tells the reader to do so. For a multi-step tutorial, preserve the files and state created by earlier steps within the documented path.

Lint before execution

The PHP CLI can check syntax without executing code. Save this complete example as hello.php:

<?php
declare(strict_types=1);

$message = 'Documentation examples should run';
echo $message, PHP_EOL;

Run:

php -l hello.php

A successful lint reports that no syntax errors were detected. A parse failure produces a nonzero status. The PHP CLI options manual describes -l as a syntax check that does not execute the file.

That distinction matters. Lint cannot establish that vendor/autoload.php exists, that an SDK class is installed, or that an API call returns the right response. It can also accept a file whose intended PHP text is outside PHP tags as plain output. The extraction script later in this guide adds an opening tag when one is absent so that a fence intended as PHP is actually parsed as PHP.

Use the PHP versions your page supports. A successful lint on a newer CLI does not establish that the same source parses on an older supported runtime. Record the version and any configuration needed to reproduce a failure.

Run complete examples in a clean directory

For a dependency-free example, start with an empty directory, copy the file there, and run it:

work_dir="$(mktemp -d)"
trap 'rm -rf "$work_dir"' EXIT
cp hello.php "$work_dir/hello.php"
cd "$work_dir"
php hello.php

The expected output is:

Documentation examples should run

A scratch directory exposes reliance on files that exist only in the author's working tree. It does not automatically isolate PHP configuration or installed runtime capabilities. Record the PHP version and relevant settings separately, and supply only the files the documentation tells the reader to create.

For examples that need a package, a clean directory must also include the documented installation step. Do not make a check pass by copying an unexplained vendor directory into place. If the page assumes a preexisting application, say so and test in a representative application; if it promises a new project, start with a new project.

Check output with assertions

Save this separate complete file as output_check.php:

<?php
declare(strict_types=1);

$actual = strtoupper('hello');
assert($actual === 'HELLO', 'Unexpected result');
echo $actual, PHP_EOL;

Run it with assertions explicitly enabled:

php -d zend.assertions=1 output_check.php

The expected output is HELLO. With assertions enabled, a false assertion raises an assertion error. PHP can be configured so assertions are not executed, so the command sets zend.assertions=1 for this check. The PHP assert() manual describes the behavior of that setting.

Choose assertions that correspond to promises in the page. For a deterministic calculation, exact equality is useful. For a response with changing identifiers or timestamps, assert the stable fields and explain the variable fields to readers. If a page shows exact output, decide whether that exactness is part of the promise before weakening a comparison.

Assertions do not replace a test of the documented command. Run the same entry point and setup a reader is told to use. If a test adds configuration that the example needs, either document that configuration or explain why the test's setting is only part of the verification.

Test Composer instructions in a scratch project

Composer instructions have their own failure modes: the package name, version constraint, PHP compatibility, lock file, repository configuration, and autoload step can each affect the result. The Composer basic usage guide describes require, composer install, the lock file, and vendor/autoload.php.

The following is a conceptual composer.json for the fictional Acme SDK. acme/acme-sdk is a fictional package name, so this is not a working installation command and should not be used to claim that a real package resolves.

{
  "require": {
    "acme/acme-sdk": "^1.0"
  }
}

For a real SDK, create a fresh scratch directory, write the actual documented composer.json, and run composer install there. If a lock file is supplied, composer install uses it; decide whether your documentation promises the locked dependency set or a fresh resolution from the stated constraints. Record the resulting versions and the PHP version.

A complete SDK example commonly needs the generated autoloader:

<?php
require 'vendor/autoload.php';

// Continue with the installed package's documented API.

This file requires a Composer installation that has created vendor/autoload.php; it is not a standalone standard-library example. The comment is intentionally a placeholder for the real package's code. For an actual page, include the appropriate namespace and use statements, then execute the complete example against the intended package version.

Test the installation and the code together. A source file may be correct against an SDK already installed on a developer's machine while the published package name or constraint is wrong. Conversely, a successful install does not prove the API calls in the next block are current.

Separate configuration from accidental local state

If an example depends on a PHP setting, identify the setting and its required value. Test with that value explicitly and, where useful, test what happens without it. The goal is to make the reader's prerequisite visible, not to assume a particular machine's configuration.

Likewise, distinguish a documented fixture from an accidental file. A tutorial may reasonably ask the reader to create an input file in an earlier step. A clean test should follow that step in order. If the page never explains how to obtain the file, the failure is a prompt to improve the instructions.

Extract and lint PHP fences

The following Python 3 standard-library script extracts simple triple-backtick php fences from one Markdown file. It prepends <?php when the block does not already start with that tag, writes each block to a temporary file, and runs php -l. It does not execute the PHP.

Save it as lint_php_fences.py:

import pathlib
import re
import subprocess
import sys
import tempfile

if len(sys.argv) != 2:
    raise SystemExit("Usage: python lint_php_fences.py PAGE.md")

page = pathlib.Path(sys.argv[1])
text = page.read_text(encoding="utf-8")
blocks = re.findall(
    r"(?m)^```php[ \t]*\r?\n(.*?)^```[ \t]*$",
    text,
    flags=re.DOTALL,
)
failed = False

with tempfile.TemporaryDirectory() as temporary:
    root = pathlib.Path(temporary)
    for number, source in enumerate(blocks, start=1):
        if not source.lstrip().startswith("<?php"):
            source = "<?php\n" + source
        source_file = root / f"block_{number}.php"
        source_file.write_text(source, encoding="utf-8")
        result = subprocess.run(
            ["php", "-l", str(source_file)],
            check=False,
        )
        if result.returncode:
            print(f"Block {number}: lint failed")
            failed = True
        else:
            print(f"Block {number}: lint passed")

if not blocks:
    print("No simple php fences found")
if failed:
    raise SystemExit(1)

Run it with Python and the PHP CLI available:

python3 lint_php_fences.py docs/page.md

This script deliberately has a narrow scope. It handles unindented fences labeled exactly php, not every Markdown or MDX fence form. It does not understand tab groups, setup steps, or intentional broken examples. A fragment that is valid only inside a class or function may fail when parsed as a file; review the documented context before changing the page.

Prepending the tag is useful for checking PHP syntax, but it can conceal a missing tag in a block that claims to be a complete file. Review complete examples for the tag separately. A passing lint also says nothing about whether the file can load Composer dependencies or run correctly.

For a larger documentation set, maintain a manifest that identifies complete files, fragments, intentional failures, fixtures, and expected outputs. Use the manifest to select the right test rather than making a regular expression infer every author's intent.

Keep examples in a normal test suite

Some documentation examples are easier to maintain as project tests, especially when they require an SDK, fixtures, and several assertions. PHPUnit provides a place to run such examples alongside the code they describe. Keep a clear connection between the tested source and the published snippet so one cannot silently change without the other.

Select a PHPUnit release compatible with the PHP runtime under test. According to the PHPUnit supported versions page, PHPUnit 12 requires PHP 8.3 or later, and PHPUnit 13 requires PHP 8.4 or later. The PHPUnit 12.4 manual describes writing tests and expressing requirements.

A test suite can verify behavior in depth, but it should not hide the reader's path. If the documentation tells readers to install a package and run a file, also test that path in a clean project. A unit test with elaborate fixtures can pass while the README omits an autoloader or gives a wrong installation command.

Put PHP documentation checks in CI

A CI job can lint each selected fence, execute complete dependency-free examples, and run project tests for examples that need dependencies. The following is a plain GitHub Actions outline. It assumes the named files exist in the repository.

name: PHP documentation examples

on:
  pull_request:
  push:
  workflow_dispatch:

jobs:
  php-docs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      # Add a step that installs PHP 8.3 or later on the runner.
      - name: Lint PHP fences
        run: python3 lint_php_fences.py docs/page.md
      - name: Run the output check
        run: php -d zend.assertions=1 output_check.php

Supply an actual PHP installation step for your runner before using the outline. Test the minimum runtime your project supports and add other supported versions as appropriate. The outline's lint script does not run Composer or PHPUnit; add those commands in the project that contains the real dependency and test configuration.

Run relevant checks when docs, public APIs, or dependency constraints change. Before release, test the artifact intended for readers where practical. A scheduled run can reveal dependency drift, while a post-release check can verify the published install path. See GitHub Actions documentation testing and installation documentation testing for related practices.

Triage failures before assigning blame

Keep the PHP version, command, settings, dependency versions, and relevant output with each result. Those details help distinguish a page defect from an environment or test-context problem.

A fragment is not defective solely because it is not a complete file. It becomes a documentation problem when the reader is not told enough to use it in the intended context. Similarly, a package-resolution failure needs investigation before it is attributed to the package name: the runtime, constraints, and network can all matter.

Where DocsAlive fits

PHP fences are recognized but skipped by DocsAlive with the reason not executable (php); the supported stacks page lists PHP among the languages that are recognized but not executed yet. They are not linted or executed, so use php -l, php, and a suitable project test suite in your own CI for PHP coverage.

Other blocks on the same page are still handled according to their language. Shell blocks and curl requests can run; JSON and TOML blocks are syntax-checked. A shell block that invokes php in the default runner image would be reported as missing-tool, because the default image is not a PHP image. Review the skip and finding reasons alongside your own PHP results.

This division is useful when a page mixes installation commands, configuration, output, and PHP source. It gives reviewers a visible record of what the product checked and what the PHP-specific job must cover. See findings for the report categories. To see how a scan reports one of your pages, you can request a health check for a public repository you own or maintain. For the same pattern in another language, see testing Java documentation examples.

Checklist

Copy this list into a pull request or release checklist:

• [ ] Classify each PHP block as a complete file, fragment, configuration, command, or output.
• [ ] State the supported PHP version and test the minimum version.
• [ ] Check opening tags in complete files and lint PHP fences.
• [ ] Review namespace, use, and autoload requirements in the reader's context.
• [ ] Run complete examples in a clean directory with only documented files and setup.
• [ ] Enable assertions explicitly when using assert() to check output.
• [ ] Install real Composer dependencies in a scratch project using the documented constraints.
• [ ] Record relevant PHP settings and test them deliberately.
• [ ] Keep PHPUnit examples connected to the published source.
• [ ] Record fragments, fixtures, credentials, and other skipped coverage.

The sources we checked

These sources were read on November 10, 2026.

FAQ