Zarr Conventions Guidance

Appearance

Implementing: URI dispatch ​

The contract: the schema_url is the version. Every breaking or semantic change gets a new URL; the old URL keeps its old meaning forever. The reader maintains a registry mapping known URLs to handlers; unknown URLs fail.

Reader skeleton ​

python
HANDLERS = {
    "https://example.com/proj/v1/schema.json": handler_v1,
    "https://example.com/proj/v2/schema.json": handler_v2,
}

def uri_dispatch(data):
    url = data["attributes"]["zarr_conventions"][0]["schema_url"]
    handler = HANDLERS.get(url)
    if handler is None:
        return "FAIL: unknown schema_url"
    return handler(data)

The registry pattern is the entire contract. New URL = the implementer writes a new handler; until then, unknown URLs fail loudly (safe). The author's commitment is "new URL for new semantics, never the same URL with new semantics." Each handler does its own field lookup against the shape it was coded for. No runtime schema validation.

How change types hit this reader ​

before.json (v1)after.json (v2)What happens
add-optionalOKOKBoth URLs are known; v1 handler reads v1 schema, v2 handler reads v2 schema.
renameOKOKv1 handler knows proj:code; v2 handler knows proj:identifier.
retypeOKOKEach handler validates against its own schema; the type difference is per-version.
semantic-onlyOKOKThe JSON is identical across the transition, but the URL is not. Each handler applies the appropriate semantics.
add-with-semantic-shiftOKOKv2 handler knows to read proj:cell_alignment; v1 handler does not need to.

The matrix is all OK. That is the point: URI dispatch handles every catalogued change type, as long as the convention author honored the covenant.

When the author breaks the covenant ​

The covenant is simple: new URL for every breaking or semantic change. Two violations matter:

  • Author ships a breaking change at the same URL. The reader has already cached a handler for that URL with the old understanding. v1-era documents (still valid against that old understanding) still read correctly. New documents written against the new (silently changed) meaning of that URL produce silent wrong output: the reader applies the old handler's logic to data that now means something different. There is no detection signal in the JSON.
  • Author publishes a new URL but the URL pattern shifts in a way the reader does not expect. Reader gets an unknown URL and fails loudly. Safe, but the new documents are unreadable until the implementer ships an update.

The first failure mode is the one that matters. URI dispatch trades "verify the author's promise" for "verify the URL changed." That tradeoff is excellent if the author has URL discipline and catastrophic if they do not.

A practical mitigation: implementers and validators can pin the URL's expected schema content (e.g., compute a hash on first fetch and reject mismatches). That makes "author silently changed the URL's content" a loud failure rather than a silent one, at the cost of more work for both sides.

Run it ​

bash
uv run walkthroughs/03-uri-dispatch/run.py

Outcome matrix ​

                            before.json  after.json
01-add-optional             OK           OK
02-rename                   OK           OK
03-retype                   OK           OK
04-semantic-only            OK           OK
05-add-with-semantic-shift  OK           OK

Reader code ​

python
# /// script
# requires-python = ">=3.10"
# dependencies = []
# ///
"""Implementer's walkthrough for Contract 3: URI dispatch.

Illustrative only. Not for production use.

The reader maintains a registry of schema URL -> per-URL handler. Each
handler reads the document according to its URL's spec, using field
lookups and hand-coded type/shape expectations. No runtime schema
validation. Unknown URL = fail.
"""

import json
import sys
from pathlib import Path

HERE = Path(__file__).parent
FIXTURES = HERE.parent / "fixtures"

CHANGE_TYPES = [
    "01-add-optional",
    "02-rename",
    "03-retype",
    "04-semantic-only",
    "05-add-with-semantic-shift",
]
INPUTS = ["before.json", "after.json"]

EXPECTED_MATRIX = {
    "01-add-optional":            {"before.json": "OK", "after.json": "OK"},
    "02-rename":                  {"before.json": "OK", "after.json": "OK"},
    "03-retype":                  {"before.json": "OK", "after.json": "OK"},
    "04-semantic-only":           {"before.json": "OK", "after.json": "OK"},
    "05-add-with-semantic-shift": {"before.json": "OK", "after.json": "OK"},
}


def load(change: str, name: str) -> dict:
    return json.loads((FIXTURES / change / name).read_text())


def _proj_attrs(data: dict) -> dict:
    return {k: v for k, v in data.get("attributes", {}).items() if k.startswith("proj:")}


def _url(data: dict) -> str:
    return data["attributes"]["zarr_conventions"][0].get("schema_url", "")


def _corner_center_anchored(t: list[float]) -> tuple[float, float]:
    return (t[2] - t[0] / 2.0, t[5] - t[4] / 2.0)


def _handler_v1(data: dict, change: str) -> str:
    """Reads documents written under the v1 URL. Knows v1 field names and v1 semantics."""
    pa = _proj_attrs(data)
    if change == "02-rename" and "proj:code" not in pa:
        return "FAIL: malformed v1 document"
    if change == "03-retype" and not isinstance(pa.get("proj:code"), str):
        return "FAIL: malformed v1 document"
    if change == "04-semantic-only" and "proj:transform" in pa:
        # v1 semantics: center-anchored. Compute and discard; the handler "uses" it.
        _ = _corner_center_anchored(pa["proj:transform"])
    if change == "05-add-with-semantic-shift" and "proj:transform" in pa:
        _ = _corner_center_anchored(pa["proj:transform"])
    return "OK"


def _handler_v2(data: dict, change: str) -> str:
    """Reads documents written under the v2 URL. Knows v2 field names and v2 semantics."""
    pa = _proj_attrs(data)
    if change == "02-rename" and "proj:identifier" not in pa:
        return "FAIL: malformed v2 document"
    if change == "03-retype" and not isinstance(pa.get("proj:code"), list):
        return "FAIL: malformed v2 document"
    if change == "04-semantic-only" and "proj:transform" in pa:
        # v2 semantics: corner-anchored.
        _ = (pa["proj:transform"][2], pa["proj:transform"][5])
    if change == "05-add-with-semantic-shift" and "proj:transform" in pa:
        alignment = pa.get("proj:cell_alignment", "center")
        t = pa["proj:transform"]
        _ = (t[2], t[5]) if alignment == "corner" else _corner_center_anchored(t)
    return "OK"


HANDLERS = {
    "https://example.com/proj/v1/schema.json": _handler_v1,
    "https://example.com/proj/v2/schema.json": _handler_v2,
}


def uri_dispatch(data: dict, change: str) -> str:
    """Dispatch on schema_url to the matching per-URL handler."""
    handler = HANDLERS.get(_url(data))
    if handler is None:
        return "FAIL: unknown schema_url"
    return handler(data, change)


def print_matrix(matrix: dict[str, dict[str, str]]) -> None:
    col_w = max(len(c) for c in CHANGE_TYPES) + 2
    val_w = max(max(len(v) for v in row.values()) for row in matrix.values()) + 2
    header = " " * col_w + "  ".join(f"{i:<{val_w}}" for i in INPUTS)
    print(header)
    for c in CHANGE_TYPES:
        row = f"{c:<{col_w}}" + "  ".join(f"{matrix[c][i]:<{val_w}}" for i in INPUTS)
        print(row)


def main() -> int:
    matrix: dict[str, dict[str, str]] = {}
    for change in CHANGE_TYPES:
        matrix[change] = {}
        for name in INPUTS:
            data = load(change, name)
            matrix[change][name] = uri_dispatch(data, change)

    print_matrix(matrix)
    if matrix != EXPECTED_MATRIX:
        print("\nFAIL: matrix does not match documented expectation.", file=sys.stderr)
        return 1
    print("\nOK: matrix matches documented expectation.")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Commentary ​

URI dispatch handles every catalogued change type because the author's covenant carries every detection signal the reader needs: the URL itself. The reader does not have to reason about version numbers, range checks, migrators, or semver semantics; it only has to ask "do I know this URL?" The survey found this is the cleanest implementation of "fail-on-unknown" in the broader JSON-metadata ecosystem (JSON Schema dialect URIs, JSON-LD @context, Frictionless profile).

The cost is that the implementer must ship a code update for every new URL the author publishes. The reader cannot cross-version share code: each handler is independent. For a convention that ships frequent additive changes, this is more friction than Integer-major with structural pass-through, which lets a single handler tolerate additive evolution within a major.