Environmental Data Retrieval (EDR)
EDR is currently in private beta. Features and capabilities may change with time.
EDR is a standard of the Open Geospatial Consortium (OGC)
The OGC API - Environmental Data Retrieval Standard provides a family of lightweight query interfaces to access spatio-temporal data resources by requesting data at a Position, within an Area, along a Trajectory or through a Corridor.
References
These are links to OGC standards and websites:
Arraylake implements a subset of the full EDR standard, specifically the Position and Area queries. Discovery of EDR collections and capabilities is not supported at this time.
Activating EDR for Arraylake Datasets
Currently the Earthmover team is responsible to activating EDR services via an internal process. Please reach out to your customer support representative to request EDR data to be activated for a specific repo.
EDR works best when datasets are compliant with CF Conventions,
particularly around coordinates. Specifically there must be 1D coordinate variables with attributes axis: X, axis: Y, axis: Z, axis: T respectively.
EDR URL Structure
Base URL
EDR endpoints can be accessed via the following URL schema.
https://{org}.compute.earthmover.io/edr/{org}/{repo}/{group}/edr
Where
{org}is the name of your Arraylake organization{repo}is the name of the Repo{group}is the name of a group within the Repo
This URL will be called {base_url} in the following examples.
All examples use the HTTP GET protocol.
Specific examples in this guide will use the Arraylake GFS Repo. The underlying GFS dataset has the following structure:
<xarray.Dataset> Size: 3TB
Dimensions: (longitude: 1440, latitude: 721, time: 736, step: 209)
Coordinates:
* latitude (latitude) float64 6kB 90.0 89.75 89.5 ... -89.5 -89.75 -90.0
* longitude (longitude) float64 12kB 0.0 0.25 0.5 0.75 ... 359.2 359.5 359.8
* step (step) timedelta64[ns] 2kB 00:00:00 01:00:00 ... 16 days 00:00:00
* time (time) datetime64[ns] 6kB 2024-05-12T18:00:00 ... 2024-11-12T1...
Data variables:
gust (longitude, latitude, time, step) float32 639GB ...
prate (longitude, latitude, time, step) float32 639GB ...
r2 (longitude, latitude, time, step) float32 639GB ...
t2m (longitude, latitude, time, step) float32 639GB ...
tcc (longitude, latitude, time, step) float32 639GB ...
Attributes:
description: GFS data ingested for forecasting demo
General Options
- By default all variables in the dataset are returned.
Specific variables can be specified in the query string via
parameter-name=t2m,tcc - Non-spatial parameters (e.g. time) can be passed as additional fields in the query string
(e.g.
time=2024-01-01T00:00:00) - The default output format is CoverageJSON.
Other formats can be specified via the
fparameter. Options include- GeoJSON:
f=geojson - CSV:
f=csv - NetCDF:
f=nc
- GeoJSON:
Position Queries
Position queries return data for the requested coordinates using the following URL structure
{base_url}/position?coords={coords}
Coordinates are encoded as well-known text (WKT). The two supported options are
- Single points:
POINT(lng lat) - Multiple points:
MULTIPOINT((lng1 lat1),(lng2 lat2))
Coordinates should be properly url-encoded, i.e. POINT(254.7295 40.0150) becomes POINT(254.7295%2040.0150)
By default, the nearest-neighbor location is returned.
Multilinear interpolation can be activated by including the option method=interp.
Multilinear interpolation is currently not as performant as nearest neighbor. The cause for this has been identified, and a fix is in development.
Examples
Get the precipitation (prate) forecast for New York City with the initialization time June 4, 2024, 12:00:00
GET https://earthmover-demos.compute.earthmover.io/edr/earthmover-demos/gfs/timeseries/edr/position?coords=POINT(286%2040.0150)&time=2024-09-04T12:00:00¶meter-name=prate
response
{
"type": "Coverage",
"domain": {
"type": "Domain",
"domainType": "Grid",
"axes": {
"y": {
"values": [
40.0
]
},
"x": {
"values": [
286.0
]
},
"step": {
"values": [
0,
3600000000000,
7200000000000,
10800000000000,
14400000000000,
18000000000000,
21600000000000,
25200000000000,
28800000000000,
32400000000000,
36000000000000,
39600000000000,
43200000000000,
46800000000000,
50400000000000,
54000000000000,
57600000000000,
61200000000000,
64800000000000,
68400000000000,
72000000000000,
75600000000000,
79200000000000,
82800000000000,
86400000000000,
90000000000000,
93600000000000,
97200000000000,
100800000000000,
104400000000000,
108000000000000,
111600000000000,
115200000000000,
118800000000000,
122400000000000,
126000000000000,
129600000000000,
133200000000000,
136800000000000,
140400000000000,
144000000000000,
147600000000000,
151200000000000,
154800000000000,
158400000000000,
162000000000000,
165600000000000,
169200000000000,
172800000000000,
176400000000000,
180000000000000,
183600000000000,
187200000000000,
190800000000000,
194400000000000,
198000000000000,
201600000000000,
205200000000000,
208800000000000,
212400000000000,
216000000000000,
219600000000000,
223200000000000,
226800000000000,
230400000000000,
234000000000000,
237600000000000,
241200000000000,
244800000000000,
248400000000000,
252000000000000,
255600000000000,
259200000000000,
262800000000000,
266400000000000,
270000000000000,
273600000000000,
277200000000000,
280800000000000,
284400000000000,
288000000000000,
291600000000000,
295200000000000,
298800000000000,
302400000000000,
306000000000000,
309600000000000,
313200000000000,
316800000000000,
320400000000000,
324000000000000,
327600000000000,
331200000000000,
334800000000000,
338400000000000,
342000000000000,
345600000000000,
349200000000000,
352800000000000,
356400000000000,
360000000000000,
363600000000000,
367200000000000,
370800000000000,
374400000000000,
378000000000000,
381600000000000,
385200000000000,
388800000000000,
392400000000000,
396000000000000,
399600000000000,
403200000000000,
406800000000000,
410400000000000,
414000000000000,
417600000000000,
421200000000000,
424800000000000,
428400000000000,
432000000000000,
442800000000000,
453600000000000,
464400000000000,
475200000000000,
486000000000000,
496800000000000,
507600000000000,
518400000000000,
529200000000000,
540000000000000,
550800000000000,
561600000000000,
572400000000000,
583200000000000,
594000000000000,
604800000000000,
615600000000000,
626400000000000,
637200000000000,
648000000000000,
658800000000000,
669600000000000,
680400000000000,
691200000000000,
702000000000000,
712800000000000,
723600000000000,
734400000000000,
745200000000000,
756000000000000,
766800000000000,
777600000000000,
788400000000000,
799200000000000,
810000000000000,
820800000000000,
831600000000000,
842400000000000,
853200000000000,
864000000000000,
874800000000000,
885600000000000,
896400000000000,
907200000000000,
918000000000000,
928800000000000,
939600000000000,
950400000000000,
961200000000000,
972000000000000,
982800000000000,
993600000000000,
1004400000000000,
1015200000000000,
1026000000000000,
1036800000000000,
1047600000000000,
1058400000000000,
1069200000000000,
1080000000000000,
1090800000000000,
1101600000000000,
1112400000000000,
1123200000000000,
1134000000000000,
1144800000000000,
1155600000000000,
1166400000000000,
1177200000000000,
1188000000000000,
1198800000000000,
1209600000000000,
1220400000000000,
1231200000000000,
1242000000000000,
1252800000000000,
1263600000000000,
1274400000000000,
1285200000000000,
1296000000000000,
1306800000000000,
1317600000000000,
1328400000000000,
1339200000000000,
1350000000000000,
1360800000000000,
1371600000000000,
1382400000000000
]
},
"t": {
"values": [
"2024-09-04T12:00:00"
]
}
},
"referencing": []
},
"parameters": {
"prate": {
"type": "Parameter",
"observedProperty": {
"id": "unknown",
"label": {
"en": "Precipitation rate"
}
},
"description": {
"en": "Precipitation rate"
},
"unit": {
"label": {
"en": "kg m**-2 s**-1"
}
}
}
},
"ranges": {
"prate": {
"type": "NdArray",
"dataType": "float",
"axisNames": [
"step"
],
"shape": [
209
],
"values": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0000032000000373955118,
0.00000960000033956021,
0.000015999999959603883,
0.000014399999599845614,
0.000016399999367422424,
0.000029200000426499173,
0.000033200001780642197,
0.00002879999919969123,
0.0000152000002344721,
0.0000016000000186977559,
0.000003999999989900971,
0.0000024000000848900527,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.00002320000021427404,
0.0,
0.000007999999979801942,
0.0000032000000373955118,
0.0001055999964592047,
0.000756800000090152,
0.0007592000183649361,
0.00012880000576842576,
0.000039999998989515007,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
4.0000000467443897E-7,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0000064000000747910235,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.000007199999799922807,
0.0,
0.0
]
}
}
}
Get all forecast variables for both New York City and Los Angeles for a single timestep in GeoJSON format
GET https://earthmover-demos.compute.earthmover.io/edr/earthmover-demos/gfs/timeseries/edr/position?coords=MULTIPOINT((286%2040.0150),(241.8%2034.1))&time=2024-09-04T12:00:00&step=0&f=geojson
response
{
"type": "FeatureCollection",
"features": [
{
"id": "0",
"type": "Feature",
"properties": {
"pts": 0,
"SUNSD": 5263,
"gust": 3.6060333251953125,
"latitude": 40,
"longitude": 286,
"prate": 0,
"r2": 55.70000076293945,
"step": 0,
"t2m": 292.3636779785156,
"tcc": 0,
"time": "2024-09-04T12:00:00"
},
"geometry": {
"type": "Point",
"coordinates": [
286,
40
]
}
},
{
"id": "1",
"type": "Feature",
"properties": {
"pts": 1,
"SUNSD": 0,
"gust": 1.2060332298278809,
"latitude": 34,
"longitude": 241.75,
"prate": 0,
"r2": 63.20000076293945,
"step": 0,
"t2m": 296.3636779785156,
"tcc": 0,
"time": "2024-09-04T12:00:00"
},
"geometry": {
"type": "Point",
"coordinates": [
241.75,
34
]
}
}
]
}
Area Queries
Area queries can be used to retrieve all of the data within a polygon.
{base_url}/area?coords={coords}
The only valid type for {coords} is
- Polygon:
POLYGON((lng1 lat1),(lng2 lat2)…(lngN latN))
Example
Create a bounding box around the state of Maine and retrieve all data variables as a NetCDF file
GET https://earthmover-demos.compute.earthmover.io/edr/earthmover-demos/gfs/timeseries/edr/area?coords=POLYGON((288.9%2042.9,288.9%2047.5,293.1%2047.5,293.1%2042.9,288.9%2042.9))&time=2024-09-04&step=0&f=nc
Authentication
For now EDR endpoints support HTTP Basic Auth with username and password.
The Earthmover team can configure credentials for EDR endpoints on a case-by-case basis.
Authenticated queries can be made by embedding the credential username and password into the base URl as follows
https://{username}:{password}@{org}.compute.earthmover.io/edr/{org}/{repo}/{group}/edr