Skip to main content

Overview

Compute is the feature of the Earthmover platform that makes it easy to query your Arraylake data and integrate it into downstream applications via standards-compliant API endpoints. This page describes the administration and management aspects of Compute. For details on the APIs and protocols, see the following pages.

  • Environmental Data Retrieval (EDR) - allows you to extract points or areas of interest from gridded datasets and return results in JSON, CSV, or NetCDF. Ideal for simple timeseries extraction.
  • Tiles - dynamically serves map tiles suitable for use in web-based slippy maps and GIS applications
  • Web Map Service (WMS) - dynamically serves map images via the OGC WMS protocol, suitable for GIS applications, static maps, and other map application use cases.
  • DAP2 - query datasets using the widely supported DAP2 protocol.
  • openEO - describe an analysis as an openEO process graph and run it server-side next to your data, downloading only the result.
  • Zax SQL - query your data as relational tables from any Postgres or Flight SQL client.
  • Discovery and Metadata - programmatically explore available repos, refs, and datasets, inspect dataset schemas, and access interactive API docs.

In the following

  • {org} is the name of your Arraylake organization
  • {repo} is the name of the Repo
  • {branch|commit|tag} is the branch, commit, or tag within the Repo to use to fulfill the request
  • {path/to/group} is the path to group within the Repo that contains an xarray Dataset
  • {protocol} is one of the supported protocols (e.g. edr, tiles, dap).

Turning on a service​

Compute is based on the concept of services. No services are enabled automatically; they must be explicitly turned on before use:

al compute enable {org} {protocol} --sku-code {sku}

--sku-code is required and sets the node size the service runs on, for example xs.

By default the service is only accessible to those with access to the Repo being queried. Provide the --is-public flag to open access to all users without authentication. For more information, see the Authentication page

Choosing a region​

A service runs in one region. Run it in the region nearest to its data: reads from the bucket are then faster and do not cross regions. Provide --region to choose one, for example:

al compute enable {org} edr --sku-code xs --region eu-west-1

Without --region, the service runs in the default region (us-east-1). The Python client lists the available regions with services.regions(). You cannot change the region of a running service. To move it, disable it and enable it again in the new region.

A service outside the default region has the region in its hostname, for example edr-1a2b3c4d.eu-west-1.compute.arraylake.app. Use the URL that al compute list {org} reports.

Turning off a service​

al compute disable {org} {service_name}

Choose the appropriate service_name from the output of al compute list {org}.

Status of Services​

al compute list {org}