HDA API

In this page

Introduction
Authentication
Accepting the terms and conditions
Subsetting datasets
Ordering data

Introduction

The Harmonised Data Access (HDA) API allows uniform access to the whole WEkEO catalogue, including subsetting and downloading functionalities.

The HDA API is REST-based and is published at this base URL: https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker All endpoints defined below are relative to this URL.

You will find usage examples of this API in WEkEO's JupyterHub service (Sign in > Dashboard > JupyterHub).

Authentication

All calls to the HDA API require an access token, which can be obtained through a call to GET /gettoken. You'll need to provide your credentials as in the following example, where <credentials> is the base64-encoded version of the string <username>: <password> . For instance, if john is your username and 123123 your password, credentials would be am9objoxMjMxMjM=:

$ curl --request GET --header 'authorization: Basic <credentials>' https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/gettoken

The HDA API will respond with a token, valid for 1 hour.

{
  "access_token": "xxxxxxxx-yyyy-zzzz-xxxx-yyyyyyyyyyyy"
}

All other calls to the HDA API must include the access token. For example:

$ curl --request GET --header 'authorization: <access_token>' https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/querymetadata/EO:EUM:DAT:SENTINEL-3:SR_1_SRA___

Accepting the terms and conditions

Before data can be accessed, the Copernicus Terms and Conditions must be accepted. This needs to be done only once.

$ curl --request PUT --header 'accept: application/json' --header 'authorization: <access_token>' --data 'accepted=true' https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/termsaccepted/Copernicus_General_License

Subsetting datasets

Each one of the datasets in the WEkEO catalogue can be subset through multiple attributes. You can obtain the full list of subsetting attributes for a particular dataset through a call to GET /querymetadata/{datasetId}. For instance, to get the attributes for ECMWF's ERA5 dataset on pressure levels:

$ curl --request GET --header 'authorization: <access_token>' https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/querymetadata/EO:ESA:DAT:SENTINEL-1:SAR

The HDA API responds with definitions for multiple variables you can use to subset, e.g. producttype:

{
  "datasetId": "EO:ESA:DAT:SENTINEL-1:SAR",
  "parameters": {
    "boundingBoxes": [
      {
        "comment": "Bounding Box",
        "details": { "crs": "EPSG:4326", "extent": [] },
        "isRequired": true,
        "label": "Bounding Box",
        "name": "bbox"
      }
    ],
    "dateRangeSelects": [
      {
        "comment": "Sensing Start / Stop Time",
        "details": {
          "start": "2018-04-30T00:41:24Z",
          "end": null,
          "defaultStart": "2018-04-30T00:41:24Z",
          "defaultEnd": null
        },
        "isRequired": true,
        "label": "Sensing Start / Stop Time",
        "name": "position"
      }
    ],
    "multiStringSelects": null,
    "stringChoices": [
      {
        "comment": "Product Type",
        "details": {
          "valuesLabels": {
            "BS": "CARD-BS",
            "CARD-COH6": "CARD-COH6",
            "GRD": "GRD",
            "GRD-COG": "GRD-COG",
            "OCN": "OCN",
            "PLANNED": "PLANNED",
            "RAW": "RAW",
            "SLC": "SLC"
          }
        },
        "isRequired": true,
        "label": "Product Type",
        "name": "producttype"
      },
      /* ... */
    ],
    "stringInputs": null
  }
}

Based on this response, you can create a subsetting job by sending a request to POST /datarequest.

$ curl --request POST \
  --url https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/datarequest
  --header 'authorization: <access_token>'
  --header 'content-type: application/json'
  --data '{
    "datasetId": "EO:ESA:DAT:SENTINEL-1:SAR",
    "boundingBoxValues": [
      { "name": "bbox", "bbox": [1.13, 43.9, 1.53, 43.68] }
    ],
    "dateRangeSelectValues": [
      {
        "name": "dtrange",
        "start": "2020-01-01T00:00:00.000Z",
        "end": "2020-01-12T00:00:00.000Z"
      }
    ],
    "stringChoiceValues": [
      { "name": "producttype", "value": "GRD" }
    ]
  }'

This returns an initial response, indicating that the new job has started:

{
  "jobId": "VBHllenZpOOi_DirRMEmoKwiAtE",
  "status": "started",
  "results": [],
  "message": null
}

Use the jobId to poll the GET /datarequest/status/{jobId} endpoint:

$ curl --request GET --header 'authorization: <access_token>' https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/datarequest/status/VBHllenZpOOi_DirRMEmoKwiAtE

until the job has finished:

{
  "status": "completed",
  "message": "Done!"
}

At this point you can retrieve the list of results calling the GET /datarequest/jobs/{jobId}/result endpoint:

$ curl --request GET --header 'authorization: <access_token>' https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/datarequest/jobs/VBHllenZpOOi_DirRMEmoKwiAtE/result

Sample response:

{
  "content": [
    {
      "downloadUri": null,
      "filename": "S1B_IW_RAW__0SDV_20200110T174650_20200110T174723_019758_0255B5_F9FC.SAFE",
      "order": null,
      "productInfo": {
        "datasetId": "EO:ESA:DAT:SENTINEL-1:SAR",
        "product": "S1B_IW_RAW__0SDV_20200110T174650_20200110T174723_019758_0255B5_F9FC.SAFE",
        "productEndDate": "2020-01-10T17:47:23.200000Z",
        "productStartDate": "2020-01-10T17:46:50.800000Z"
      },
      "size": 1672255055,
      "url": "040ff724-6884-5366-ac68-eb96a78d2c4f/S1B_IW_RAW__0SDV_20200110T174650_20200110T174723_019758_0255B5_F9FC.SAFE"
    },
    /* ... */
  ],
  "itemsInPage": 10,
  "page": 0,
  "pages": 7,
  "previousPage": null,
  "totItems": 70
}

As you can see, WEkEO has found 70 items and is returning only the first page with 10 results. You can add size and page query parameters to your call to retrieve other results:

$ curl --request GET --header 'authorization: <access_token>' https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/datarequest/jobs/VBHllenZpOOi_DirRMEmoKwiAtE/result?size=15&page=3

Ordering data

Once a subsetting job has completed, you can issue data orders for any of the job results. To do this, create an order job by sending a request to POST /dataorder, including the jobId and the url of the result to be downloaded.

$ curl --request POST \
  --url https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/dataorder
  --header 'authorization: <access_token>'
  --header 'content-type: application/json'
  --data '{
  "jobId": "VBHllenZpOOi_DirRMEmoKwiAtE",
  "uri": "040ff724-6884-5366-ac68-eb96a78d2c4f/S1B_IW_RAW__0SDV_20200110T174650_20200110T174723_019758_0255B5_F9FC.SAFE"
}'

This returns an initial response, indicating that the new job has started:

{
  "orderId": "xwShzM5es927MUbTXBFOeka_EvA",
  "status": "running",
  "message": null
}

Use the orderId to poll the GET /dataorder/status/{orderId} endpoint:

$ curl --request GET --header 'authorization: <access_token>' https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/dataorder/status/xwShzM5es927MUbTXBFOeka_EvA

until the job has finished:

  "status": "completed",
  "message": "Done!",
  "downloadUri": "040ff724-6884-5366-ac68-eb96a78d2c4f/S1B_IW_RAW_ _0SDV_20200110T174650_20200110T174723_019758_0255B5_F9FC.SAFE",
  "url":
   "040ff724-6884-5366-ac68-eb96a78d2c4f/S1B_IW_RAW_ _0SDV_20200110T174650_20200110T174723_019758_0255B5_F9FC.SAFE"
}

At this point you can download the data calling the GET /dataorder/download/{orderId} endpoint:

$ curl --request GET --header 'authorization: <access_token>'https://wekeo-broker.apps.mercator.dpi.wekeo.eu/databroker/dataorder/download/xwShzM5es927MUbTXBFOeka_EvA

After a short while, your files should have been downloaded to your working directory.