The Iceberg REST Catalog Compatibility Test: One Suite of Operations Every Platform Should Pass

Cross-posted. This article's canonical home is iceberglakehouse.com.

"Supports the Iceberg REST catalog" is the most load-bearing claim in the modern data platform market, and it has no test. Every catalog service, warehouse, and managed platform makes it, buyers weigh entire architectures on it, and the claim's actual content, which operations, from which clients, under which policies, varies so widely that two platforms making the identical sentence true can differ on whether your engine can create a table, evolve a schema, or write a row. This site has documented that variance empirically, more than once, and documentation of variance is not the fix. The fix is a test: a defined suite of operations, run the same way against every platform, scored in a shared four-value vocabulary, published with dates attached, and re-run on a standing calendar.

This article is that test's specification, written to be implemented, and it is also a commitment: this suite, run across the catalogs practitioners actually choose between, becomes a standing, annually refreshed asset of this site, a compatibility matrix with a methodology behind it rather than a review with opinions inside it. What follows defines the design principles the suite must satisfy, the seven levels of operations it probes, from attachment to remote scan planning, the four-way scoring vocabulary that keeps policy refusals distinct from failures, the runner architecture that keeps it engine-agnostic, and the publication protocol that keeps it honest over time. If you run a platform, this is the checklist your documentation should be able to answer. If you are choosing one, this is the checklist to run before the contract gets signed.

Disclosure, stated with unusual care given the subject: I work at Dremio, whose catalog offerings are built on Apache Polaris and will appear in this suite's published results like everyone else's, scored by the same runs, with the methodology public precisely so that the scoring cannot quietly favor anyone. I co-authored O'Reilly's books on Apache Iceberg and Apache Polaris. The suite tests behavior, and behavior does not care who signs my paychecks.

Why a Test Suite, and Not Another Review

The genre this article refuses deserves a paragraph, because the refusal is the design.

Platform reviews and comparison posts, this site's included, decay in three ways. They age silently, accurate at publication and misleading eighteen months later, with no mechanism that flags the decay. They blur two different things, what the specification's protocol supports and what a given platform's policy permits, into one impression of "compatibility," when the distinction is the single most decision-relevant fact in the territory. And they resist verification, since a prose claim about what happened during someone's testing cannot be re-run by a reader, a vendor, or the author's future self. Earlier work on this site, the seven-catalog write-probe protocol run through a single embedded engine, was built to escape the third decay, publish the method, not just the findings, and its reception taught the obvious next lesson: a method worth publishing for one engine is worth generalizing into a suite for any engine, and a suite worth running once is worth institutionalizing on a calendar.

A test suite escapes all three decays structurally. Dated, versioned runs make staleness visible instead of silent, this matrix reflects suite version so-and-so run in such-and-such month, and the previous matrix remains published beside it, so the decay is a diff anyone can read. A scoring vocabulary that separates refusal from failure keeps the spec-versus-policy distinction in the results themselves rather than in caveats nobody reads. And a public test runner makes every cell reproducible: disagree with a result, run the cell, file the discrepancy with logs attached, which converts arguments about compatibility from testimony into evidence.

There is also a quieter reason the suite matters now, and it is the trajectory this site has traced all year. The population of things that speak to catalogs is exploding, embedded engines, native libraries in four languages, services, agents, and every new client multiplies the pairings whose behavior somebody has to know. Pairwise experiential knowledge, the veteran engineer who remembers which platform refuses which operation, does not scale to that combinatorics, and the arithmetic is worth one sentence: eight platforms by five clients by forty operations is sixteen hundred cells, a number no team's institutional memory holds and a machine re-verifies in an afternoon. Official conformance work, the fixture and test efforts inside the project, addresses the specification's side of the question, whether implementations read and write the format correctly. The gap between those two, how commercial platforms' policies and integrations actually behave at their REST boundaries, is exactly the gap a practitioner-run suite fills, complementary to the official work, adjacent to it, and honest about being a different thing: conformance tests ask "is this implementation correct," this suite asks "what will this platform let your tools do."

Design Principles

Six principles govern every decision below, stated up front because the levels and scoring are their consequences.

Reproducible above all. Every cell in the published matrix corresponds to a scripted operation with captured output, runnable by anyone with credentials to the platform in question, and no cell ever encodes an impression, a support ticket's promise, or a documentation claim. If it did not execute, it is not in the matrix.

Engine-agnostic by architecture. The suite defines operations semantically, create a namespace, append rows, evolve a schema, commit concurrently, and a driver layer translates each operation into a given engine's dialect, so the same suite runs through an embedded engine, a JVM engine, or a native library, and the matrix gains a dimension: platform by operation by client, because this site's core finding is precisely that the client changes the answer.

Policy-aware, never policy-blind. A refusal is a result, not a bug, and the scoring vocabulary treats a platform that cleanly rejects external writes as reporting its governance model, not failing a test. The suite's job is to make the policy surface visible and comparable, which respects both the platforms that choose openness and the platforms that choose control, and flatters only the ones whose documentation matches their endpoints.

Dated and versioned, both the suite and the runs. Operations get added as the ecosystem grows, scan planning did not exist to test three years ago, and every published matrix names the suite version and run window, because a compatibility claim without a date is the exact disease this project treats.

Cheap enough to actually re-run. The whole suite against one platform completes in minutes, not days, on free tiers and trial accounts where possible, with small fixtures, because a test that costs real money and real weeks gets run once and becomes a review with extra steps. Cheapness is what makes the annual cadence, and vendor re-verification, and reader reproduction, realistic rather than aspirational.

And bounded in scope, deliberately. The suite probes functional compatibility at the REST boundary, what operations succeed, refuse, or fail. It does not benchmark performance, judge pricing, or evaluate operational qualities, uptime, support, ecosystem, that a real selection also weighs, and the matrix's introduction says so every year, because a good instrument states what it does not measure.

What the Suite Excludes, and Why the Exclusions Are Load-Bearing

The boundedness principle deserves its own section, because each exclusion protects the instrument's trustworthiness, and readers should know the protections are deliberate.

Performance is excluded because it is unownable at this boundary. A latency number for an operation blends the platform, the client, the network path, the region, the tier, and the fixture size, and publishing one invites every wrong comparison the blend enables, while the operational qualities that matter, plan latency under metadata mass, commit latency under contention, deserve dedicated methodology with controlled variables, a different instrument entirely. The one performance-adjacent datum the suite keeps is categorical: the L4 note on whether commit latency stays flat under contention, recorded as a behavior, server-side reconciliation present or absent, never as a milliseconds figure.

Pricing and packaging are excluded because they change faster than any annual instrument tracks and because the suite's neutrality depends on never ranking value, only recording behavior. The matrix tells you what a platform's boundary does, and what that is worth against its price is exactly the judgment that belongs to the buyer holding their own workload, their own ownership cards, and this suite's facts.

Maintenance quality is excluded with genuine regret, because this site has argued all year that who compacts, and how well, decides table health, and the exclusion is forced by observability: a platform's optimization service works on its schedule, against its heuristics, over weeks, and a minutes-long suite run cannot fairly sample it. What the suite can and does capture is the adjacent boundary fact, whether external maintenance is possible, the L2 and L3 cells covering rewrites and property changes from outside, which tells an operator whether they can own dimension three themselves when the platform's automation disappoints.

And spec conformance of the platform's underlying implementation is excluded by jurisdiction: whether an engine's Parquet writing, delete application, or lineage materialization is bit-correct belongs to the project's conformance fixtures and interop tests, run at the implementation layer with the format's own machinery, and this suite deliberately sits above that layer, consuming its results rather than duplicating them. The division of labor keeps both instruments sharp, and the escalation path between them real: a suite fails cell whose logs implicate the underlying implementation gets filed upstream, jurisdiction observed, ecosystem improved.

The exclusions compose into the suite's one-sentence identity, worth stating because instruments drift without one: this is the behavioral map of the platform boundary, nothing more, and everything a behavioral map can be.

The Seven Levels

The suite organizes its operations into levels, ordered so that each level's operations presume the previous level's success, an ordering which gives every platform a natural high-water mark, the highest level at which its cells are green, and gives readers a one-glance summary before descending into the cell-by-cell detail.

Level 0, attachment and discovery. Can a client establish a session at all: resolve the endpoint, complete the platform's authentication flow, load catalog configuration from the config endpoint, list namespaces, list tables, and load a table's metadata. Level 0 sounds trivial and is where real friction lives, because authentication is the least standardized surface in the ecosystem, OAuth flows, static tokens, cloud-signature schemes, per-platform header conventions, and a platform whose Level 0 requires undocumented incantations has told you something the rest of the matrix should be read in light of. The level also captures the attachment recipe itself as an artifact, the exact configuration that worked, which is, for practitioners, the single most reused output of the whole suite, and it records one forward-looking cell besides: what the config endpoint advertises, the capability and override properties returned, because the protocol's capability negotiation is the mechanism through which every future level, planning today, whatever v4 requires tomorrow, gets discovered, and a platform's advertisement accuracy, does it claim what it serves and serve what it claims, is a small cell with outsized predictive value for the rest of the row.

Level 1, the read path. With a session established: full-table scan of a small fixture table, filtered read that should engage metadata pruning, projection of a column subset, time travel to a prior snapshot, and reads of the standard metadata tables, snapshots, files, manifests, where the client exposes them. Level 1 is where format-version reach gets probed too, reading a v2 fixture, then a v3 fixture bearing deletion vectors, then one with lineage columns requested, because "reads Iceberg" and "reads current Iceberg" have diverged, and the divergence belongs in cells rather than footnotes.

Level 2, DML writes. The heart of the matrix, and the direct descendant of this site's earlier probe: create a namespace, create a table, append rows, read them back through the same session, then the row-level operations, UPDATE, DELETE, MERGE, then explicit small-batch appends simulating a modest streaming cadence, and finally DROP, with the result recording what DROP did, registration removal or data deletion, an ownership fact this site has argued belongs in every evaluation. Level 2 is where the four-way scoring earns its keep, because this is the level where platform philosophy shows: the open-catalog genre tends green across the row, the warehouse genre tends toward refusals on external writes into managed namespaces, and both patterns are findings. Two refinements sharpen the level's row-level cells. The delete cells record the mechanism produced, not just the outcome, whether the platform's path wrote copy-on-write rewrites, position artifacts, or v3 deletion vectors, because a green delete cell that produced artifacts half your reader fleet cannot consume is a finding the mechanism column catches and the outcome column misses. And the format-version dimension from Level 1 recurs here as a write question, whether tables get created at v2 or v3 by default and whether the version is selectable, since the create-cell's default quietly decides which of the format's current features every subsequent cell exercises.

Level 3, evolution. Schema changes through the client: add a column, rename a column, widen a type, drop a column, then partition spec evolution, and a table-property change, including the property flips that matter operationally, write modes, retention settings. Evolution is where integrations shear, a platform can accept appends happily and refuse or mishandle schema changes arriving from outside its own tooling, and pipelines discover that shear at the worst possible time, which is why it gets its own level rather than a footnote to writes.

Level 4, the concurrency probe. Two sessions, same table: concurrent appends from two clients, which should both land through retry, a concurrent append during a long write, and, where the platform permits row-level writes, a genuine conflict, two operations touching the same rows, which should produce a clean validation failure on one side rather than corruption or a hang. The level tests the platform's implementation of the commit protocol's requirements under contention, including whether retryable and semantic failures surface distinguishably, the taxonomy this site's concurrency coverage argues every operator needs, and a bonus cell records whether the platform reconciles non-conflicting commits server-side, visible as flat commit latency under contention. L4 is also where the suite verifies the invariant no platform is permitted to trade away: after every contention cell, a history check confirms the table's snapshot log is linear with both writes' effects present or one cleanly absent, because a forked history discovered here, in a fixture table, is the finding that justifies the entire level's existence, and the one cell where fails triggers immediate vendor contact rather than waiting for the notice window.

Level 5, governance and credential vending. Where the platform implements them: whether scoped storage credentials are vended per table and honored by the client, whether a deliberately under-privileged principal is refused cleanly at the operations its grants exclude, whether refusals arrive as intelligible errors naming the policy rather than as opaque failures, and whether audit surfaces record the suite's operations attributably. Level 5 is scored with special care because it inverts the usual valence: here, refusals in the right places are the passing behavior, and a platform that lets the under-privileged principal write has failed a cell by succeeding at an operation. The vending cells carry one probe beyond mere function: scope verification, whether a credential vended for table A actually fails against table B's paths, because vending that hands out broader access than it claims is the assumed-policy failure mode this site's ownership coverage describes, wearing the costume of a governance feature, and it is exactly the kind of gap that only an adversarial cell, attempting what should be impossible, ever finds.

Level 6, remote scan planning. The frontier level, added this suite version, testing the 1.11-era capability this site has covered in depth: does the platform serve the planning endpoints, does a capable client negotiate and use them, do planned scans return correctly filtered tasks, do policy constraints, the row and column controls of Level 5, bind within planned results, and does a client without planning support fall back gracefully to self-planning where policy permits. Level 6 cells are expected to be sparse for a while, capability negotiation is young, and sparse cells with dates are exactly how a matrix teaches an ecosystem's trajectory, the same way delete-support cells taught v2's adoption curve.

The levels, compressed to their essence:

| Level | Question it answers | Representative operations | |---|---|---| | L0 | Can a client attach and look around? | Auth, config, list namespaces and tables, load table | | L1 | Can it read, including current-format features? | Scans, filters, time travel, metadata tables, v3 fixtures | | L2 | Can it change data? | Create, append, update, delete, merge, drop semantics | | L3 | Can it change structure? | Schema evolution, partition evolution, properties | | L4 | Does concurrency behave? | Concurrent appends, genuine conflict, failure taxonomy | | L5 | Does governance bind? | Vending, under-privileged refusals, audit attribution | | L6 | Does planning serve? | Plan endpoints, filtered tasks, policy in plans, fallback |

The Scoring Vocabulary

Every cell takes exactly one of four values, and the vocabulary is the suite's most important design decision, carried over and hardened from this site's earlier protocol.

Works: the operation succeeded through the standard client path with standard configuration, output verified, no platform-specific ceremony beyond documented attachment.

Works with configuration: the operation succeeded and required platform-specific, documented steps beyond the standard path, a feature flag, a privileged connection mode, a property the platform requires, recorded verbatim in the cell's notes, because "possible with the right settings" and "possible" are different answers to a buyer and the matrix preserves the difference.

Refused by policy: the platform rejected the operation deliberately, with an error indicating authorization or governance rather than malfunction, the platform functioning as designed under a policy that excludes the operation for this principal or namespace. Refusals are findings with full dignity: a column of refusals is a governance model, legible, comparable, and for some buyers exactly what they want.

Fails: the operation errored in a way that is neither success nor intelligible refusal, a crash, a hang, a corruption, a success response with wrong results, the last being the gravest and specifically probed for, since wrong-answers-quietly is the failure mode this site's format coverage keeps flagging as worse than any error. Fails cells carry logs, and, per the publication protocol below, a vendor notice before the matrix ships.

The vocabulary's discipline is refusing a fifth value. No "partial," no "planned," no "works in beta," because fuzzy values are where matrices go to become marketing, and every tempting middle case maps to one of the four with notes: the beta feature that worked is works-with-configuration noted as beta, the operation that succeeded with wrong results is fails, full stop.

The gravest classification deserves its probe described, because "success response with wrong results" is not caught by checking return codes. Wherever the suite writes, it verifies content: the appended rows re-read and compared, the deleted rows confirmed absent, the merged state checked against the expected outcome computed independently, and Level 1's format probes extend the same discipline to reads, the v3 fixture with deletion vectors carrying known live-and-deleted row sets, so a reader that ignores vectors and returns deleted rows as live, the exact wrong-answers failure this site's delete coverage flags as the ecosystem's worst mode, converts a green-looking read into a fails cell with the discrepancy in the log. Content verification is most of the runner's actual code and all of its credibility, since a compatibility matrix that trusts status codes is measuring politeness rather than correctness.

One Cell, Walked Through

Abstractions land better with one concrete execution, so trace a single Level 2 cell, "append rows via the embedded-engine driver," against a composite platform, end to end.

Attachment reuses the run's Level 0 session: the profile supplied the endpoint, an OAuth client-credentials flow, and one platform-specific property the profile's notes flag as required, which has already marked this platform's row as works-with-configuration territory at L0. Setup creates the cell's fixture table in the suite's permitted namespace through the driver, a four-column table, and the creation itself was the previous cell, green, so this cell starts from a real, empty, suite-owned table.

Execution issues the append, one hundred rows through the driver's dialect, and the driver captures everything: the statement, the client-side logs, the commit's round trips. Verification then does the independent re-read, first through the same driver, then through the run's second attached client, the Python library, confirming both see one hundred rows with matching content, and a metadata check confirms exactly one new snapshot of operation type append landed, no surprise extra commits, which has caught real integration quirks before, retry loops double-committing under a misconfigured timeout.

Classification consults the taxonomy: clean success, standard path, no extra ceremony beyond the profile's documented attachment property, and the cell scores works, with the run-level note that attachment required configuration living at L0 where it belongs rather than repeated in every subsequent cell. Teardown drops the fixture table and, because this platform's earlier DROP-semantics cell recorded registration-plus-deletion, verifies the storage cleanup happened, one more free datum confirming an earlier cell's finding.

Now the same cell against a second composite platform, the warehouse genre: attachment green, creation green in the platform's external-write-permitted namespace, and the append into the platform's managed namespace comes back with an authorization error naming the namespace's write policy. The error's shape matches the taxonomy's refusal patterns, the cell scores refused-by-policy with the error text in notes, and the pair of runs has just reproduced, mechanically, dated, with logs, the central finding of this site's earlier manual protocol, which is exactly the point of having built the machine.

Runner Architecture

The suite's implementation shape, specified tightly enough to build and loosely enough to survive contact with real platforms.

Three abstractions carry the whole design. A catalog profile describes one platform target: endpoint, authentication method and materials, warehouse or account identifiers, the namespace the suite is permitted to use, and any platform-required configuration, the profile being exactly the attachment recipe Level 0 validates and publishes. An engine driver implements the suite's operation vocabulary for one client: given "append rows to table T," the embedded-engine driver issues its SQL dialect, the library driver calls its API, the JVM-engine driver submits its statement, each driver small because the operation vocabulary is small, a few dozen verbs. And a run binds one profile to one driver, executes the levels in order, and emits results, machine-readable cells plus captured logs, which the reporting layer renders into the published matrix.

The operation flow per cell is uniform and boring on purpose: set up the cell's fixture state, execute the operation through the driver, verify the outcome through an independent read where possible, classify into the four-way vocabulary using the error taxonomy, refusal-shaped errors versus failure-shaped ones, record everything, tear down. In sketch:

def run_cell(profile, driver, operation):
    ctx = attach(profile)                  # L0 machinery, reused per run
    fixture = operation.setup(ctx)         # tables/rows the op needs
    try:
        result = driver.execute(ctx, operation, fixture)
        verified = operation.verify(ctx, result)
        return Cell(WORKS if verified.standard_path
                    else WORKS_WITH_CONFIG,
                    notes=verified.notes, log=result.log)
    except RefusalError as e:              # auth/policy-shaped errors
        return Cell(REFUSED_BY_POLICY, notes=e.policy_hint, log=e.log)
    except Exception as e:                 # everything else
        return Cell(FAILS, log=capture(e))
    finally:
        operation.teardown(ctx, fixture)

Four implementation decisions round out the architecture. Fixtures are minimal and self-created: the suite provisions its own tiny tables inside its permitted namespace, a few hundred rows, both v2 and v3 variants for Level 1's format probes, so executions need nothing pre-staged and clean up after themselves. Verification prefers independence: after a write through driver A, the runner re-reads through the catalog's own interface or a second driver where available, because a client that believes its own write is weaker evidence than a second path confirming it. Error classification is a maintained taxonomy, the mapping from each platform's error shapes to refusal-versus-failure, and it is the runner's most platform-specific component, versioned with the suite, because a misclassified refusal is the likeliest way for the matrix to be wrong. And everything lands in a repository, profiles minus secrets, drivers, operations, taxonomy, and the rendering pipeline, because the reproducibility principle is a repository or it is rhetoric.

The engine-driver dimension deserves its own emphasis, since it is what the earlier single-engine protocol lacked and what the findings demanded. The suite's first-class matrix is three-dimensional, platform by operation by client, published as one two-dimensional matrix per client, and the initial client set follows the population this site has tracked: an embedded analytical engine, the Python library, a JVM batch engine, with the native Rust and Go libraries joining as their write paths mature, each addition a driver rather than a rewrite. The cross-client comparison is where the suite's deepest findings will live, the operation that works from the JVM engine and fails from the library against the same platform being precisely the integration reality practitioners plan around and no single-client test can see.

The Publication Protocol

A matrix is trustworthy in proportion to the process around it, so the process is part of the specification.

Each published edition names the suite version, the run window, the exact platform tiers and client versions tested, and every profile's non-secret configuration. Cells link to logs. The edition ships with a changelog against the previous edition, cells that changed value, operations added, platforms added or retired, because the diffs are half the value: a works that became a refusal is a policy change somebody's pipeline met in production, and a fails that became a works is an ecosystem improving, both worth a practitioner's minute.

Vendors get a notice window, not a veto. Before publication, each platform's fails cells go to the platform with logs, with a defined window to respond, and responses land in the cell's notes, a fix shipped and verified re-runs green, a configuration correction reclassifies the cell, a disputed classification publishes with both readings. The window converts the matrix from ambush into accountability, and the no-veto rule keeps it journalism rather than co-marketing.

Cadence is annual for the full matrix, with event-driven partial re-runs, a major platform release, a new suite level, a disputed cell, refreshing affected rows between editions, each partial re-run dated in place. The annual rhythm is a promise this article makes on the record: the matrix's value compounds only if edition two exists, and the graveyard of one-time comparison posts is the alternative this protocol exists to avoid.

Reader reproduction gets first-class support, because the reproducibility principle owes readers more than a repository link. Each edition ships a reproduction guide per platform: the trial-tier or free-tier path to credentials where one exists, the profile template to fill, the one command that runs a row, and the expected wall-clock and cost, minutes and cents for most platforms. A reader verifying a cell that matters to their decision is the system working as designed, a reader finding a different result is a dated discrepancy report the next partial re-run investigates, and the difference between a matrix people cite and a matrix people trust is exactly this loop running a few dozen times a year.

And the suite itself evolves in public: operations proposed as issues, additions versioned, the level structure stable but not frozen, Level 6 being this version's admission that the frontier moves, and the next frontier, multi-table transaction endpoints, richer planning capabilities, v4-era metadata behaviors when they ship, already visible enough to sketch as future levels.

Reading a Matrix Row: Three Archetypes

The matrix teaches interpretation as much as facts, so here are the three row patterns readers will meet, described as archetypes because the patterns are structural, recurring across vendors within each genre.

The open-catalog pattern reads green down the levels through L3, both concurrency cells clean at L4, vending and refusal cells green at L5, and an L6 that depends on which implementation lineage and version the platform runs, with the works-with-configuration notes concentrated at L0, since open catalogs vary most in their authentication ceremonies. The interpretation: a platform positioning itself as neutral infrastructure, external clients as first-class citizens, and the evaluation weight for a buyer shifts entirely to the dimensions this suite deliberately excludes, operations, maintenance offerings, the ownership card's other cells, because compatibility itself is answered.

The warehouse pattern reads green across L0 and L1, then splits at L2 into the pattern the walkthrough showed: green in externally-writable namespaces, refused-by-policy in managed ones, with L3 typically following L2's split, L5 strong, often the row's best level, since governance is the genre's center of gravity, and L6 arriving on the platform's own schedule. The interpretation is the one this site has insisted on all year: this row is a governance philosophy, not a deficiency, the platform telling you that its managed tables are written through its paths, and the buyer's question is whether that philosophy matches their intended topology, external writers or platform-mediated ones, a question the row answers precisely and no adjective answers at all.

And the integration-tier pattern, the row that motivates the suite's existence, reads unevenly: L1 green, L2 mixed with works-with-configuration cells whose notes run long, an L3 cell or two red, L4 untested-or-fails, the signature of a platform whose REST surface is an adapter over an older architecture, real but partial, improving edition over edition. The interpretation is temporal: this row is a trajectory, compare it against last edition's, and the diff tells you whether the platform is investing in the boundary or maintaining it, which is, for a multi-year commitment, the more decision-relevant fact than any single cell.

Three patterns, and the meta-lesson the archetypes carry: rows are read as wholes, against genres and against their own history, and the matrix's cells are evidence for that reading rather than isolated verdicts, which is why every edition ships with this interpretive guide attached, updated as the patterns themselves evolve.

What the Matrix Is For

The artifact's uses, in ascending order of ambition, because the design choices above serve all four.

Selection: a buyer choosing a catalog platform reads their client's matrix row-set against their required operations and gets, in one table, the answer that currently takes weeks of trials to assemble, with the works-with-configuration notes doubling as the setup guide for whatever they choose. The levels' high-water summary serves the executive version of the same conversation.

Regression detection: teams already running a platform diff editions to learn what changed under them, the policy tightened, the operation newly supported, before their pipelines learn it the hard way, which turns the annual edition into something between a changelog and an early-warning service for the ecosystem's integration surface. The partial re-run mechanism serves the same use at higher frequency for the platforms a team actually depends on: subscribe to a row, in effect, by re-running it quarterly with your own credentials against your own tier, the reproduction guide making that an hour's automation, and the suite's operation vocabulary doubles as the smoke test a platform migration or version upgrade runs before production traffic does, which is the suite quietly becoming operational tooling rather than merely published research, an outcome the repository's structure deliberately invites.

Accountability: the claim this article opened with, "supports the Iceberg REST catalog," acquires a public referent, and the gap between a platform's marketing and its matrix row becomes legible to everyone, which is the gentlest available pressure toward documentation that matches endpoints and policies stated rather than discovered. The four-way vocabulary keeps the pressure fair, platforms are never punished for governance choices, only for opacity about them.

And convergence: the suite feeds the larger movement this site has repeatedly pointed at, the conformance fixtures, the interop test efforts, the multi-implementation era's shared verification infrastructure. A practitioner-run behavioral suite at the platform boundary complements spec-conformance work at the implementation boundary, findings from one flowing into the other, a platform's fails cell occasionally being an upstream implementation's bug surfaced through a commercial wrapper, and the long-run success condition is worth stating plainly: this suite should eventually be boring, every cell green or cleanly refused, editions differing only as new levels probe new frontiers, because the variance it was built to map has been engineered away. Instruments that aspire to their own obsolescence tend to be the trustworthy ones.

An Invitation, and the Commitment Restated

The suite described here is being built in the open, and three kinds of participation make it better. Practitioners: run cells against platforms you have credentials for, especially tiers and regions the main runs miss, and file discrepancies with logs, because coverage is the resource a single author cannot scale alone. Platform teams: implement the checklist against your own endpoints before the run does, publish your own results if you like, the methodology is yours to use, and treat the notice window as the collaboration channel it is meant to be. And toolmakers: drivers are small, the operation vocabulary is documented, and every client added multiplies the matrix's value for that client's users.

Governance questions arrive with contribution, and the answers are set now, before they are needed under pressure. Operation additions and taxonomy changes go through public proposal, with the standard being decision-relevance to practitioners rather than any platform's convenience, and additions that advantage or disadvantage specific vendors get argued in the open where the argument itself is a record. Disputed classifications resolve by re-execution, never negotiation: the cell is whatever a fresh run with published configuration produces, both parties watching, which is the tidiest dispute mechanism ever invented and the reason the reproducibility principle sits first. And the conflict-of-interest handling for my own employer's rows is structural rather than promissory: same suite version, same drivers, same notice window, logs published like everyone's, plus a standing invitation, renewed in every edition, for any third party to re-run those rows specifically and publish alongside. An instrument's neutrality is not an author's virtue, it is an architecture that makes the author's virtue unnecessary, and this specification has tried to be that architecture throughout.

The commitment, restated as the article closes, because institutionalization is the whole point: the first full edition runs against the platforms practitioners actually shortlist, publishes on this site with everything the protocol above requires, and returns annually, dated, versioned, diffed, and reproducible. The ecosystem earned a real answer to its most load-bearing claim. This is the instrument, and the instrument is now on the calendar.

Conclusion

The Iceberg REST catalog made compatibility claimable in one sentence, and only a test makes the sentence mean something. Seven levels, from attachment to remote scan planning, cover what practitioners actually need to know, four scoring values keep platform policy distinct from platform failure, an engine-driver architecture keeps the suite honest about the finding that started all of this, that the client changes the answer, and a publication protocol, dated, versioned, vendor-noticed, annually renewed, keeps the artifact trustworthy in the only way artifacts stay trustworthy, structurally. Reviews describe an ecosystem, tests improve one, and the difference between the two is everything this specification exists to build. Every article this site has published this year, on scan planning, on writes, on deletes, on ownership, generated questions this suite converts from prose into cells, which is the deepest sense in which it is the site's signature asset: it is the body of work, made executable. The claim finally gets its test. Run it, reproduce it, extend it, and hold every platform, this author's employer included, to the same seven levels.

Keep Going

If this piece was useful, I have written a lot more on Apache Iceberg and lakehouse architecture. Apache Iceberg: The Definitive Guide and Apache Polaris: The Definitive Guide, which I co-authored for O'Reilly, cover the catalog protocol, governance, and architecture this suite probes. You can find every book I have written, across lakehouse architecture, Apache Iceberg, Apache Polaris, and AI, at books.alexmerced.com.