# Compatibility tracker methodology

The public tracker at [macmake.dev/compatibility](https://macmake.dev/compatibility/) is generated from [`data/compatibility.json`](../data/compatibility.json). Update that data and its public-safe evidence reports; do not edit the generated HTML directly. Each observation has an explicit `evidence_origin`: `official_release`, `release_candidate`, or `development_build`. A release-candidate observation names an unpublished archive and must never be described as the current public release. The matrix's public-release status remains tied to v0.1.0 until a later binary is published. Candidate and development observations do not upgrade that status. Active engineering work may be recorded as a separate development activity before it has produced a compatibility result.

## Evidence levels

- `runtime_verified`: the named application produced by the identified macMAKE build was observed loading, starting, and performing its stated functions on that exact OS version and architecture. This confirms that application's build path for the tested system; it does not claim that every application or Toolbox API works.
- `startup_verified`: loading and startup were observed, but functional testing was limited.
- `build_verified`: the identified release compiled, linked, packaged, and structurally validated the application; OS runtime was not verified.
- `compilation_verified`: relevant source compiled, but a complete usable application was not verified.
- `not_tested`: available evidence is insufficient to classify the OS/version and architecture combination as compatible.
- `unsupported`: the release lacks a required capability or a specific limitation is established.

“Testing in progress” is a separate activity flag and never upgrades a compatibility status. Build or PEF validation on Linux does not establish that a Macintosh operating system loads or runs the application.

## Add a test result

1. Test with the exact public macMAKE release and record its version, executable or archive digest, provider identity, target OS/version, and architecture. Label development builds explicitly.
2. Record the fixture/application, authorized SDK/runtime requirements, test date (or leave it null when the retained evidence cannot establish one), and a concise public-safe evidence report.
3. Record compilation, linking, packaging, OS loading, startup, and functional behavior separately. Do not infer a later gate from an earlier one.
4. Classify the result at the highest demonstrated level. A successful runtime test demonstrates that macMAKE produced an application which runs on the tested system, within the behavior recorded for that fixture. A small application such as makePAD establishes basic application support. Claims about networking, cloud services, email, browsers, or other advanced behavior require corresponding application evidence. The matrix names each runtime-tested application.
5. Add the evidence report under `compatibility/evidence/`, update the target's `observations`, and update `status` and `current_release_status` consistently. Keep development and release-candidate observations clearly labeled; do not upgrade `current_release_status` unless the exact public release binary was tested.
6. Run:

   ```sh
   python3 scripts/build-compatibility-page.py
   python3 scripts/check-site.py --macmake /path/to/the-exact-release/macmake
   ```

   To check generated HTML without rewriting it, run `python3 scripts/build-compatibility-page.py --check`.

The validator rejects unknown statuses, duplicate target identifiers or OS/architecture pairs, missing release/toolchain/fixture identities, invalid dates, missing evidence files, unsupported entries with positive test claims, gate/status contradictions, and generated HTML drift. A target may contain `development_activity` to summarize ongoing unreleased work; it must be marked as outside v0.1.0 and is not a compatibility result. A target may also contain `attempts`: these record blocked or failed runs, identify the failed stage and blocker, and do not raise the compatibility status. This keeps a source or environment failure visible without mislabeling it as an OS compatibility result. Public evidence must not include private source, SDK contents, credentials, server configuration, or unnecessary local paths.

## Runtime handoff

The owner operates Classic Mac emulators and physical Macs. Linux-side work may prepare and structurally inspect a MacBinary artifact, record its hash, and transfer it using the established owner-controlled workflow. Do not mark OS loading, startup, or functional behavior passed until the owner reports the actual observation. Never operate the owner's QEMU session.

## Planned earlier-PowerPC test sequence

After the public-release MacSurf rebuild and owner-side Mac OS 9 runtime handoff, begin with small C89 fixtures. The initial order is System 7.5.5, Mac OS 8.1, System 7.6.1, Mac OS 8.6, System 7.1.2, Mac OS 8.0, Mac OS 8.5, then Mac OS 9.0 and 9.1 where they add coverage. Availability of an authorized SDK, startup code, and runtime may change the practical order; record why when it does.

Use three progressively larger fixtures:

1. **Minimal application:** startup/termination, required CFM imports, resources, Finder metadata, and native PowerPC packaging.
2. **Toolbox window:** Window Manager, QuickDraw, a basic event loop, and clean termination.
3. **Interactive application:** a menu or button, dialog/alert, user interaction, cleanup, and exit.

Select interfaces that exist on the exact OS version. Do not add CarbonLib to fixtures for pre-Carbon systems or assume Mac OS 9 runtime libraries exist on earlier OS versions. Record SDK, headers, MSL/startup files, and imports used for each result.

Classify every failure at the observed layer: source, manifest/project configuration, SDK availability, compiler, ABI, linker/import resolution, runtime/startup, resources/package, OS loader, or application behavior. A failed later gate must not erase successful earlier evidence, and passing one small fixture does not establish broad API compatibility.

## Keep 68K separate

The public v0.1 release does not provide native 68K application generation. As of 2026-10-16, an unreleased macMAKE-controlled GNU 68K target has built HelloWindow, which the owner confirmed launches, displays its message, and supports window dragging on Mac OS 8.1. The same target built all six makePAD source files. The first makePAD candidate launched and accepted text input, but the owner reported broken window dragging and File-menu interaction plus a glitched scrollbar. A replacement using the verified HelloWindow event-polling sequence and desktop-wide drag bounds has passed host build and structural checks; its owner runtime retest is pending. These development results do not change v0.1 support. The current application path uses 68000-family code generation, macMAKE-owned startup and Toolbox veneers, classic CODE resources, and MacBinary packaging. Its current one-segment implementation uses PC-relative application globals; it is not yet a general A5-relative global-data or multi-segment runtime. Earlier OS compatibility still requires exact-version runtime tests, and System 6 and early System 7 may expose additional instruction, API, memory, and loader requirements.

## Owner runtime handoff

After host-side PEF/resource/Finder/MacBinary inspection passes, record and transfer one explicitly selected MacBinary artifact using the owner's documented SSH reverse tunnel and `~/.ssh/macsurf_push` key. Verify the destination hash over the same tunnel. The owner unpacks and launches it in the OS guest and reports the exact version, observed startup and behavior, and any failure. Do not start or control the guest.