Implementing: Semver with declared migrators

The contract: full MAJOR.MINOR.PATCH versioning, with an Arrow-style compatibility-range promise and a declared migrator for each release. The reader implements the latest known version, range-checks every incoming document, and applies the documented migrator to normalize older data into the latest shape before processing.

Where the migrator lives is a separate architectural choice: it may live in the convention's spec text (as algorithmic guidance), or in a designated reference library (the way pystac carries STAC's migrators while the stac-spec repo only ships changelogs). Other implementations either depend on the reference library or port its migrators. The contract works the same either way; the burden distribution differs.

Reader skeleton

def semver_with_migrators(data):
    version = parse_semver(url(data))
    if not in_compat_range(version):
        return "FAIL: out of compatibility range"

    pa = proj_attrs(data)
    if version < latest_known:
        pa, transformed = MIGRATORS[version](pa)        # spec-shipped rule

    return read_latest(pa)                              # the v2 path

The migrator is the contract's distinctive piece. The convention (or its reference library) must declare one for every release that changed shape or semantics, so any in-range document can be normalized into the latest representation in memory before the reader's v2 path consumes it. After migration the reader does the same field-lookup-and-use it would do natively; no runtime schema validation is needed.

How change types hit this reader

	before.json (v1)	after.json (v2)	What happens
add-optional	OK	OK	v1 data validates against the v2 schema (the new field is optional). No migration needed; reader proceeds via the v2 path.
rename	MIGRATED	OK	The spec's migrator renames proj:code to proj:identifier. After migration, the v2 reader sees v2 shape.
retype	MIGRATED	OK	The migrator parses the string EPSG code into the integer array. v2 reader proceeds.
semantic-only	MIGRATED	OK	The migrator shifts transform coefficients by half a pixel to convert center-anchored to corner-anchored. After migration, the v2 reader applies v2 semantics and produces the right answer.
add-with-semantic-shift	MIGRATED	OK	The migrator inserts proj:cell_alignment: "center" (matching v1's implicit default). v2 reader then dispatches on the field correctly.

All MIGRATED-or-OK. The contract handles every catalogued change type if the author ships the migrator. The matrix's strength is that the v2 reader has one code path; only the migrators differ per release.

When the author breaks the covenant

Semver with migrators has the strictest covenant of the five: the author commits to shipping a working migrator for every release, declaring an honest compatibility range, and maintaining semver discipline across releases. Three violations:

• Author bumps a minor or patch without shipping a migrator. The reader has no migration path for the older shape; it either falls back to fail-loudly (refuse the data) or processes naively (and silently misreads, depending on how the implementer wrote the no-migrator path). Reader correctness now depends on the implementer's fallback choice.
• Author claims a compatibility range that is too wide. v1 data falls in-range; the migrator does not actually fit the case (e.g., the rename migrator doesn't know about a third intermediate name from a deprecated 0.x release). After migration, the data is wrong-shaped, and v2 schema validation either fails loudly or — if the migrator's bug produced a coincidentally-valid shape — passes with silently wrong values.
• Author ships a major bump. Reader fails loudly with FAIL: out of compatibility range. Safe; the reader knows it cannot handle this data and refuses cleanly.

The major failure mode is migration debt: the contract works only as long as the author ships migrators every release, and the implementer keeps up. STAC plus pystac is the catalogued reference case where this discipline is maintained over years; the survey notes that stac-spec's 1.0 → 1.1 transition included three tighten rows that produced documents invalid against v1.0 schemas, which the migrators absorb. The contract is excellent when the maintainers operate it consciously.

Run it

uv run walkthroughs/05-semver-with-migrators/run.py

Outcome matrix

                            before.json  after.json
01-add-optional             OK           OK
02-rename                   MIGRATED     OK
03-retype                   MIGRATED     OK
04-semantic-only            MIGRATED     OK
05-add-with-semantic-shift  MIGRATED     OK

Reader code

# /// script
# requires-python = ">=3.10"
# dependencies = []
# ///
"""Implementer's walkthrough for Contract 5: Semver with declared migrators.

Illustrative only. Not for production use.

The reader implements v2 (the latest known version). For documents
declaring an older version inside the compatibility range, it applies
an in-spec migrator that transforms the v1 shape into v2 shape in
memory, then processes through the v2 path. Out-of-range = fail.
No runtime schema validation.
"""

import json
import re
import sys
from pathlib import Path

HERE = Path(__file__).parent
FIXTURES = HERE.parent / "fixtures"

CHANGE_TYPES = [
    "01-add-optional",
    "02-rename",
    "03-retype",
    "04-semantic-only",
    "05-add-with-semantic-shift",
]
INPUTS = ["before.json", "after.json"]

EXPECTED_MATRIX = {
    "01-add-optional":            {"before.json": "OK",       "after.json": "OK"},
    "02-rename":                  {"before.json": "MIGRATED", "after.json": "OK"},
    "03-retype":                  {"before.json": "MIGRATED", "after.json": "OK"},
    "04-semantic-only":           {"before.json": "MIGRATED", "after.json": "OK"},
    "05-add-with-semantic-shift": {"before.json": "MIGRATED", "after.json": "OK"},
}


def load(change: str, name: str) -> dict:
    return json.loads((FIXTURES / change / name).read_text())


def _proj_attrs(data: dict) -> dict:
    return {k: v for k, v in data.get("attributes", {}).items() if k.startswith("proj:")}


def _parse_major(data: dict) -> int | None:
    url = data["attributes"]["zarr_conventions"][0].get("schema_url", "")
    m = re.search(r"/v(\d+)/", url)
    return int(m.group(1)) if m else None


def _migrate_v1_to_v2(pa: dict, change: str) -> tuple[dict, bool]:
    """Per-change migrator shipped by the convention's spec. The reader applies
    it in memory to normalize v1 data to v2 shape before processing.
    Returns the (possibly transformed) attrs plus a flag indicating whether
    any actual transformation was applied.
    """
    pa = dict(pa)
    if change == "02-rename":
        if "proj:code" in pa:
            pa["proj:identifier"] = pa.pop("proj:code")
            return pa, True
    elif change == "03-retype":
        code = pa.get("proj:code")
        if isinstance(code, str):
            m = re.match(r"EPSG:(\d+)", code)
            if m:
                pa["proj:code"] = [int(m.group(1))]
                return pa, True
    elif change == "04-semantic-only":
        t = pa["proj:transform"]
        pa["proj:transform"] = [t[0], t[1], t[2] - t[0] / 2.0, t[3], t[4], t[5] - t[4] / 2.0]
        return pa, True
    elif change == "05-add-with-semantic-shift":
        pa.setdefault("proj:cell_alignment", "center")
        return pa, True
    return pa, False


def _read_v2(pa: dict, change: str) -> None:
    """v2 reader path. Looks up fields by their v2 names and applies v2 semantics."""
    if change == "02-rename":
        _ = pa.get("proj:identifier")
    elif change == "03-retype":
        _ = pa.get("proj:code", [None])[0]
    elif change == "04-semantic-only":
        t = pa["proj:transform"]
        _ = (t[2], t[5])  # corner-anchored
    elif change == "05-add-with-semantic-shift":
        t = pa["proj:transform"]
        if pa.get("proj:cell_alignment", "center") == "corner":
            _ = (t[2], t[5])
        else:
            _ = (t[2] - t[0] / 2.0, t[5] - t[4] / 2.0)


def semver_with_migrators(data: dict, change: str) -> str:
    """Range-check the version. In-range v1 data is migrated to v2 in memory
    via the spec-shipped migrator before processing. The reader implements
    v2 only.
    """
    major = _parse_major(data)
    if major not in {1, 2}:
        return "FAIL: out of compatibility range"

    pa = _proj_attrs(data)
    transformed = False
    if major == 1:
        pa, transformed = _migrate_v1_to_v2(pa, change)

    _read_v2(pa, change)
    return "MIGRATED" if transformed else "OK"


def print_matrix(matrix: dict[str, dict[str, str]]) -> None:
    col_w = max(len(c) for c in CHANGE_TYPES) + 2
    val_w = max(max(len(v) for v in row.values()) for row in matrix.values()) + 2
    header = " " * col_w + "  ".join(f"{i:<{val_w}}" for i in INPUTS)
    print(header)
    for c in CHANGE_TYPES:
        row = f"{c:<{col_w}}" + "  ".join(f"{matrix[c][i]:<{val_w}}" for i in INPUTS)
        print(row)


def main() -> int:
    matrix: dict[str, dict[str, str]] = {}
    for change in CHANGE_TYPES:
        matrix[change] = {}
        for name in INPUTS:
            data = load(change, name)
            matrix[change][name] = semver_with_migrators(data, change)

    print_matrix(matrix)
    if matrix != EXPECTED_MATRIX:
        print("\nFAIL: matrix does not match documented expectation.", file=sys.stderr)
        return 1
    print("\nOK: matrix matches documented expectation.")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Commentary

Semver with migrators is the most robust contract on the menu and the most expensive to operate. It is the only contract that handles semantic-only and add-with-semantic-shift cleanly across the version transition: the migrator normalizes old data into new shape before processing, so the v2 reader's single code path serves every supported release. The cost is that the author or reference-library maintainers must write the migrators, the convention community must point implementers at them, and every reader must apply them. If any link breaks, the contract degrades to the same failure modes as Integer-major with structural pass-through or worse. STAC has held this together for years by concentrating migration work in pystac and letting other consumers ride that maintenance, which is the lowest-burden version of the contract that still actually works.

Choose this contract when (a) your maintainer community is engaged enough to write migrators with every release, and (b) your domain demands the ability to read older data after semantic shifts. If either condition is uncertain, prefer the lower-cost contracts and accept their failure modes.