Iceberg REST Catalog Reference

Arraylake natively implements the Apache Iceberg REST Catalog specification. This page documents the base URL, authentication, credential vending, supported endpoints, and commit semantics. Most users never call these endpoints directly — engines do — but this reference is useful when configuring an engine not covered in Connecting Query Engines, or when debugging.

Base URL

https://api.earthmover.io/iceberg

All catalog routes live under this URI, e.g. GET https://api.earthmover.io/iceberg/v1/config.

Authentication

Every request must carry an ordinary Arraylake bearer token:

Authorization: Bearer ema_...

Both user OAuth tokens and api-client tokens (ema_...) are accepted; api-client tokens are recommended for engines because they don't expire mid-session.

warning

The deprecated Iceberg OAuth2 flow (POST /v1/oauth/tokens) is not supported. Configure clients with a static token, not a credential (client id/secret) pair.

Warehouse and prefix resolution

The warehouse identifier is your organization name. There is exactly one catalog per org and it always exists — it is never created, and it is not tied to a bucket. Catalog clients begin a session by fetching the configuration:

GET /iceberg/v1/config?warehouse=my-org

{
  "defaults": {},
  "overrides": {
    "prefix": "my-org",
    "warehouse": "my-org"
  },
  "endpoints": ["GET /v1/config", "GET /v1/{prefix}/namespaces", "..."]
}

The returned prefix is the org name, so all subsequent requests take the form /iceberg/v1/{org}/.... The endpoints array advertises exactly which routes the catalog supports (listed below); spec-compliant clients disable any capability not listed.

A warehouse identifier containing a / is a 400: it is an org name, not org/bucket. An unknown org is a 404 (NotFoundException) before any authorization detail is revealed, and a caller with no visibility anywhere in the org's catalog gets a blanket 403.

Namespaces

A namespace is a first-class Arraylake entity: it belongs to an org, and it is bound to one bucket configuration plus a server-generated prefix in it, which is where its tables are stored. Tables are separate entities inside it.

Namespaces are flat. Identifiers carry exactly one namespace level, and tables are addressed <namespace>.<table>. A nested reference (raw.obs) is rejected with a 400 BadRequestException — "nested namespaces are not supported" — rather than being silently flattened. A nested table identifier is refused as 404 NoSuchTableException instead, because engines (Spark's metadata tables) probe table-shaped identifiers speculatively and need that shape to fall back correctly. GET /v1/{prefix}/namespaces?parent=... with a non-empty parent returns an empty list, since nothing nests.
listNamespaces lists the namespaces you can see in the org — those you hold a namespace grant on, plus any namespace holding a table you were granted individually. A caller with no visibility at all gets a 403 rather than an empty list.
createNamespace creates one on the org's default bucket, unless the request properties name another bucket configuration with arraylake.bucket: <nickname>. It requires the org's Iceberg feature flag and org-level write standing; it generates and collision-checks a storage prefix, verifies that prefix is empty, and writes an .arraylake-warehouse.json marker there. Request properties are stored on the namespace; a location property is defaulted to its storage root (Trino requires one). Recreating a namespace whose soft-deleted, still-empty predecessor holds the name restores that one in place.
dropNamespace soft-deletes the namespace — the name and prefix stay claimed for a 7-day recovery period and it can be restored, with its grants, through the management API or the web app. After that, Arraylake empties the namespace's storage prefix and removes it; see Deleting and restoring namespaces for the immediate and retain_data options, which are available through the management API and the Python client but not through the catalog. As the spec requires, a namespace that still contains tables cannot be dropped: 409 NamespaceNotEmptyException.
Namespace names follow Arraylake's repo-name rules (start with a letter, digit, or underscore, then one or more of word characters, ., -, _; at most 128 characters). Invalid names are rejected with a 400. Avoid . — engines split identifiers on dots, so such a namespace would look nested and be rejected.
Namespaces and repos share one set of names per organization. A namespace cannot take the name of a repo in the same org, and a repo cannot take the name of a namespace — case-insensitively, and including soft-deleted ones that are still recoverable. Both sides answer 409 (AlreadyExistsException from the catalog; a name_held_by_iceberg_namespace detail from the repo API). This keeps every name in an org unambiguous wherever repos and namespaces appear side by side, such as the web app and SQL tooling that lists both as schemas of one catalog.
Renaming a table across namespaces is not supported (400): a table's files live under its namespace's prefix (or its own override), so moving the catalog pointer elsewhere would leave the data behind. Renaming within the namespace works.

Table storage overrides

A table normally lives at <namespace prefix>/<table>/ on the namespace's bucket. A single table can instead bind its own storage by naming a bucket configuration in the arraylake.bucket table property on createTable:

catalog.create_table(
    "my-tables.big_events",
    schema=schema,
    properties={"arraylake.bucket": "my-other-bucket"},
)

The table then claims a freshly generated, empty prefix on that bucket, checked against the same collision guard as namespaces. Prefixes are always server-generated — there is no arraylake.prefix property. Two restrictions apply:

The override cannot be combined with a staged create (stage-create, e.g. PyIceberg's create_table_transaction), which persists no row to carry the binding: that returns a 400.
An explicit location in the request must still fall inside the table's own storage (its override prefix, or the namespace's location/prefix); anything else is a 400.

Buckets configured for anonymous (unauthenticated, public-read) access cannot back Iceberg storage and are rejected with a 400: the catalog must be able to write table metadata and vend credentials. Azure buckets are not yet supported.

Access control

Namespaces default to PRIVATE. A namespace admin can set visibility to AUTHENTICATED_PUBLIC from the namespace's settings in the web app or with modify_iceberg_namespace in the Python client (or an org admin can set it when creating a namespace; see Manage namespaces with Python). This lets any authenticated Arraylake user or API client discover the namespace and read all its tables, without org membership or explicit grants. It grants no write or administration permissions. Anonymous access remains disabled.

Public namespaces require customer-managed-role authentication on their bucket and any table bucket overrides, so storage credentials can be scoped to read-only access. Static HMAC credentials are not supported for public namespaces. Changing visibility back to PRIVATE removes public access on subsequent requests; already-issued storage credentials remain valid until they expire. Explicit grants remain in effect. Soft-deleted namespaces are not publicly discoverable.

Grants are made at two grains, to the same principals as the rest of Arraylake — users, teams, and api-clients:

Grant	Covers
Namespace grant	the namespace and every table in it, now and in future
Table grant	that one table, by name

Three actions are grantable, each implying the ones below it:

Read — list a namespace's tables, load table metadata and namespace properties, receive read-scoped storage credentials.
Write — create/drop/rename tables and commit to them, update namespace properties, drop the namespace. Also implies read.
Manage users — administer the namespace's grants (including table grants inside it). Also implies write and read.

Table grants are bound to the table's name. They survive a table being dropped and recreated under the same name, and renameTable carries them along to the new name (merging with any grant that already sat on that name), so a rename never changes who can reach the data.

How the two grains interact:

A namespace grant cascades to all of its tables.
A table grant covers that table and, so that it is reachable at all, makes its parent namespace visible: the namespace appears in listNamespaces and can be loaded, but nothing else in it is readable — listTables returns only the granted tables, and every other table in the namespace is a 403.
Writing at the namespace grain (dropping the namespace, updating its properties, creating a table) requires a namespace grant. A single-table grant is never authority over the container.
Table grants are bound to the table name, so they survive a drop-and-recreate of that table. Revoking is always explicit.

Organization roles bypass the grant tables entirely, mapping down onto every namespace in the org: an org member who can read repos can read all namespaces and tables; one who can write repos can write them; an org admin who can manage users can administer Iceberg grants everywhere. Earthmover service administrators have full access.

Grants are administered from the namespace's access-control tab in the web app, where a grant can be scoped to the whole namespace or to one table. Team grants require the Teams feature on your plan.

Credential vending

When a client sends the header

X-Iceberg-Access-Delegation: vended-credentials

on LoadTable or CreateTable requests, the response includes temporary storage credentials scoped to the table's own location — never to the namespace prefix or the whole bucket — valid for one hour. That matches the table-level grain of access control: a principal holding one table grant never receives credentials covering the rest of the namespace. PyIceberg and DuckDB send this header automatically; Spark must be configured to send it (see Connecting Query Engines).

Credentials are returned in both forms defined by the spec, for maximum client compatibility:

the storage-credentials array, each entry carrying a prefix and a config map;
duplicated into the flat config object of the LoadTableResult (s3.access-key-id, s3.secret-access-key, s3.session-token, or gcs.oauth2.token / gcs.oauth2.token-expires-at for GCS).

Both forms are a stable part of this API's contract. Some clients only consume the flat config keys — iceberg-rust deserializes and then discards the structured storage-credentials array, so for Rust writers the flat keys are the entire vending mechanism. Vending works on CreateTable as well as LoadTable, so writers have credentials before their first load.

Temporary credentials carry their expiry where the platform reports one (s3.session-token-expires-at-ms, gcs.oauth2.token-expires-at), letting engines schedule a refresh instead of failing mid-scan.

For S3-compatible buckets the config also carries s3.endpoint, s3.path-style-access, and s3.region so clients need no storage configuration at all.

Refreshing credentials

Vended credentials expire after one hour. Clients that hold a table open longer than that — long scans, or writers whose jobs outlive the vend — should refresh rather than reload:

GET /iceberg/v1/{org}/namespaces/{namespace}/tables/{table}/credentials

returns a fresh storage-credentials array (same shape and scoping as the load-time vend). This endpoint is advertised in GET /v1/config; PyIceberg refreshes automatically on table reload, while clients managing their own FileIO are responsible for calling it before expiry, using s3.session-token-expires-at-ms to schedule the refresh.

Long-running writes with iceberg-rust

iceberg-rust (as of 0.9) uses credentials as delivered and never re-fetches — it ignores the expiry key and this endpoint. Jobs that finish within the one-hour credential lifetime are unaffected; a single write that outlives it will fail mid-job unless the application layers in its own credential-refreshing storage.

Supported endpoints

These are the routes defined by the Iceberg REST Catalog specification, not an Arraylake-specific API: engines call them directly, and spec-compliant clients disable any capability the catalog does not advertise, so knowing which subset Arraylake implements is what lets you configure or debug an engine. Request and response shapes are the spec's own; see the OpenAPI definition for those. Arraylake's own management endpoints for namespaces (creation options, description and metadata, visibility, grants, delete and restore) are not documented as raw HTTP routes, the same as for repositories: use the Python client and CLI or the web app.

The catalog advertises the following endpoints in GET /v1/config ({prefix} is the {org}, and {namespace} is a single namespace level):

Endpoint	Description
`GET /v1/config`	Catalog configuration; resolves the warehouse (org) to a prefix
`GET /v1/{prefix}/namespaces`	List the namespaces you can see in the org
`POST /v1/{prefix}/namespaces`	Create a namespace
`GET /v1/{prefix}/namespaces/{namespace}`	Load a namespace and its properties
`HEAD /v1/{prefix}/namespaces/{namespace}`	Check namespace existence
`DELETE /v1/{prefix}/namespaces/{namespace}`	Drop an (empty) namespace — soft-deletes it
`POST /v1/{prefix}/namespaces/{namespace}/properties`	Update namespace properties
`GET /v1/{prefix}/namespaces/{namespace}/tables`	List the tables you can see in a namespace
`POST /v1/{prefix}/namespaces/{namespace}/tables`	Create a table (including staged create)
`GET /v1/{prefix}/namespaces/{namespace}/tables/{table}`	Load a table (with optional credential vending)
`GET /v1/{prefix}/namespaces/{namespace}/tables/{table}/credentials`	Refresh vended storage credentials
`HEAD /v1/{prefix}/namespaces/{namespace}/tables/{table}`	Check table existence
`POST /v1/{prefix}/namespaces/{namespace}/tables/{table}`	Commit updates to a table
`DELETE /v1/{prefix}/namespaces/{namespace}/tables/{table}`	Drop a table
`POST /v1/{prefix}/tables/rename`	Rename a table within its namespace
`POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/metrics`	Report scan metrics (accepted and discarded)
`GET /v1/{prefix}/namespaces/{namespace}/views`	List views — always empty (views are otherwise unsupported)

Notes:

Namespaces are single-level (see above). The spec's 0x1F unit separator is still parsed on the wire, but any identifier carrying more than one namespace level is rejected.
Pagination on list endpoints is opt-in per the spec: send pageSize to receive paged results with a next-page-token; without it, all results are returned in a single page.
Dropping a table removes it from the catalog immediately and its data files stay in storage: purgeRequested is accepted (the drop still returns 204, since engines treat a DROP TABLE ... PURGE error as a failed drop) but nothing is deleted. See Storage is not reclaimed.

Commit and conflict semantics

Iceberg's REST protocol splits responsibility between engine and catalog. The engine writes data files and manifests directly to object storage (using vended credentials); the Arraylake server then:

validates the commit's requirements (e.g. "the table's current snapshot is still X") against the table's current metadata;
applies the metadata updates and writes the new metadata.json to storage;
atomically swaps the table's metadata pointer (compare-and-swap).

If two writers race, exactly one commit wins. The loser receives the spec's 409 with type: CommitFailedException, e.g.:

{
  "error": {
    "message": "Table has been updated by another process: my-tables.observations",
    "type": "CommitFailedException",
    "code": 409
  }
}

This is a retryable conflict: compliant engines (PyIceberg 0.12+, Spark, DuckDB, Trino) automatically re-read the latest table metadata, rebase the commit, and retry. Concurrent writers therefore work out of the box; no locking or coordination is needed on your side. PyIceberg releases before 0.12 surface the conflict as a CommitFailedException without retrying, so refresh the table and re-apply the write yourself on those versions.

Snapshot summary properties: authorship and lineage

Custom snapshot summary properties pass through commits untouched, are preserved in the table's snapshot history, and are displayed in the Arraylake web app (expand any row in a table's snapshot history).

Two kinds of keys are reserved under the arraylake. prefix:

Stamped by the server on every added snapshot (caller-supplied values for these keys are overwritten — attribution comes from the authenticated principal, not the client):

Key	Value
`arraylake.author-name`	The committing principal's display name (users only)
`arraylake.author-email`	The committing principal's email

Lineage convention — writers that derive Iceberg tables from other Arraylake data (e.g. point extraction from an icechunk dataset) should stamp the source into each snapshot's summary so results stay traceable and reproducible:

Key	Value
`arraylake.source.repo`	Source repo, as `org/repo`
`arraylake.source.ref`	Source branch or tag name
`arraylake.source.snapshot-id`	Source icechunk snapshot id
`arraylake.source.query`	The query that produced the data — a serialized logical plan, or a stable hash of one

These are conventions, not validated by the server today; tooling (lineage links in the web app) will build on them.

arraylake.bucket is not a snapshot key

arraylake.bucket is a namespace/table creation property read once to choose storage (see Table storage overrides), not a snapshot summary key.

Error model

Errors are rendered as the spec's ErrorModel:

{
  "error": {
    "message": "Namespace does not exist: my-tables",
    "type": "NoSuchNamespaceException",
    "code": 404
  }
}

Code	Type	Meaning
400	`BadRequestException`	Malformed request; a warehouse containing `/`; nested namespace reference; invalid namespace name; anonymous-access bucket; unknown bucket nickname in `arraylake.bucket`; `arraylake.bucket` combined with stage-create; table location outside the table's storage; cross-namespace rename; malformed table updates
403	—	Missing namespace or table grant, or no visibility anywhere in the org's catalog (standard Arraylake access denied)
403	`NotAuthorizedException`	Iceberg is not enabled for the organization
404	`NotFoundException`	Unknown warehouse — no such org
404	`NoSuchNamespaceException`, `NoSuchTableException`	Namespace or table does not exist
409	`AlreadyExistsException`	Namespace or table already exists, or its storage prefix is taken or not empty
409	`NamespaceNotEmptyException`	Dropping a namespace that still contains tables
409	`CommitFailedException`	Concurrent commit conflict — rebase and retry

Unsupported features

The following parts of the Iceberg REST spec are not supported in this release:

Views. The one exception is listing (GET .../namespaces/{namespace}/views), which is advertised and always returns an empty list so engines' view probes succeed; creating, loading, and dropping views are not available.
Format-version 3 tables: creating a table with format-version: 3 (or upgrading to it) is rejected with a 400 until the server can produce fully valid v3 metadata (apache/iceberg-python#1551). Existing v2 tables are unaffected.
Nested (multi-level) namespaces — namespaces are flat by design; see Namespaces.
Moving a table between namespaces with renameTable (the data would be left under the source namespace's prefix). Renaming within a namespace is supported.
Multi-table transactions (POST /v1/{prefix}/transactions/commit)
Registering existing tables (POST /v1/{prefix}/namespaces/{namespace}/register)
Namespace levels and table names containing / or the 0x1F namespace separator are rejected with a 400 at creation/rename: such names could never be addressed through the REST API's URL paths afterwards.
Server-side scan planning
The deprecated OAuth2 token endpoint (POST /v1/oauth/tokens)
Azure Blob Storage buckets. Credential vending currently maps to the s3.* and gcs.* engine config keys only; ADLS credential vending (adls.*) is not yet implemented, so namespaces and table overrides must use an S3 or GCS bucket. (This matches Icechunk's existing Azure gaps.)
Synchronous purge on drop, and storage reclamation generally — see below.

Storage is not reclaimed

Arraylake does not run snapshot expiration or garbage collection for Iceberg tables. Files that stop being referenced — expired snapshots' manifests and data files, rewritten manifests, the files of a dropped table — remain in the bucket. DELETE ...?purgeRequested=true (e.g. Spark's DROP TABLE ... PURGE) drops the catalog entry but deletes nothing. Until Iceberg maintenance ships, reclaiming that space is a manual, bucket-side operation; the Icechunk repo optimization settings do not apply to Iceberg.

The exception is a deleted namespace: once its recovery period elapses (or immediately, when deleted with immediate), everything under its storage prefix is removed, including the files of tables that were dropped from it. Pass retain_data to keep those bytes.

Base URL​

Authentication​

Warehouse and prefix resolution​

Namespaces​

Table storage overrides​

Access control​

Credential vending​

Refreshing credentials​

Supported endpoints​

Commit and conflict semantics​

Snapshot summary properties: authorship and lineage​

Error model​

Unsupported features​

Storage is not reclaimed​