Export API

Table of Contents

Overview

• Overview

Exports

• Get an export of a repository version
• Create an export of a repository version
• Determine the status of an export file
• Delete an export file

Other

• Useful HTTPie commands

Overview

The API provides an export endpoint for creating, fetching, and deleting specific repository versions. Requesting an export returns a status of 200 OK with the exportURL in the response header if the file already exists, 102 Processing if it is being processed, or a 204 No Content if the export does not exist or is still being processed. The filename of the export includes the lastUpdated timestamp of the export, which can be used to fetch the diff between an export and the current state of a source or collection. The exportURL is designed to be used only once -- to ensure that the exportURL works correctly, you must request another exportURL each time you want to download an export file.

Exports are a compressed tar of the JSON results. The JSON results are the equivalent of the following REST API requests:

GET /orgs/:org/sources/:source/:sourceVersion/?includeConcepts=true&includeMappings=true&includeRetired=true&limit=0
GET /orgs/:org/collections/:collection/:collectionVersion/?includeConcepts=true&includeMappings=true&includeRetired=true&limit=0

The export filename takes the following form:

[:repositoryId]_[:repositoryVersion].[:lastUpdated].tar

To fetch the diff between a lastUpdated timestamp and the current state of a source or collection:

GET /orgs/:org/sources/:source/?includeConcepts=true&includeMappings=true&includeRetired=true&limit=0&updatedSince=:lastUpdated
GET /orgs/:org/collections/:collection/?includeConcepts=true&includeMappings=true&includeRetired=true&limit=0&updatedSince=:lastUpdated

Refer to sources and collections documentation for details on the above requests.

The Subscriptions documentation describes how the export functionality can be used to subscribe to a source or collection to keep a client system in synch.

Future Considerations

• Add lastUpdated into the response header of GET request (right now lastUpdated is only stored in the filename)

Get an export of a repository version

• Get the export URL for the specified repository version. This has three possible results:
  • If the export exists, returns a status code of 200 OK and with exportURL in the response header, containing the fully specified URL to download the export file
  • If the export file does not exist or is still being processed, returns a status code of 204 No Content and exportURL is empty
  • If the export request is invalid or not authorized, returns the error code

GET /[:ownerType/]:owner/:repoType/:repo/:repoVersion/export/

• Notes
  • repoVersion is required - exports can only be created for repository versions
  • The API returns the fully specified URL of the export file in the exportURL attribute of the response header. The exportURL is designed to be used only once -- to ensure that the exportURL works correctly, you must request another exportURL each time you want to download an export file.
  • This request first performs a check as to whether the export file already exists. If it exists, it then generates an exportURL. These actions are performed on an Amazon Web Server and may take up to 30 seconds to process. The timeout of the client system making the request should set its timeout period accordingly.

Example

• Fetch the export URL for v2016-08-22 of the CIEL source

GET /orgs/CIEL/sources/CIEL/v2016-08-22/export/

• Fetch the export URL for v1.2 of the CIEL Starter Set

GET /orgs/CIEL/collections/StarterSet/v1.2/export/

Response

• If the export file is cached:

Status: 200 OK
Response Header:
exportURL: https://ocl-source-export-production.s3.amazonaws.com/CIEL/CIEL_55572f688a86f24b48d935be.20150516122820.tgz?Signature=fmBSI6hL9IhN4mu4W5x%2FPFs5uxw%3D&Expires=1431864368&AWSAccessKeyId=...

• If the export file does not exist:

Status: 204 No Content
Response Header:
exportURL: 

• If the export file is still being prepared:

Status: 102 Processing
Response Header:
exportURL: 

• If the requested export is invalid - returns the appropriate error code

Create an export of a repository version

• Create an export file for the specified repository version; if it already exists, no action is taken

POST /[:ownerType/]:owner/:repoType/:repo/:repoVersion/export/

• Notes
  • :repoVersion is required
  • This request only triggers the creation of the export file and does not return the exportURL. It is necessary to follow up with a GET request after the file has been processed in order to get the exportURL

Example

• Create the export file for v2.2 of the CIEL source

POST /orgs/CIEL/sources/CIEL/v2.2/export/

Response

• If no export file already exists and processing is initiated:

Status: 202 Accepted

• If an export file is currently being processed:

Status: 102 Processing

• If the export file already exists

Status: 204 No Content

• If the request is otherwise invalid - return the appropriate error code

Determine the status of an export file

• Returns the current status of an export file. Output is identical to the GET request.

HEAD /[:ownerType/]:owner/:repoType/:repo/:repoVersion/export/

Delete an export file

• Deletes a specific export file

DELETE /[:ownerType/]:owner/:repoType/:repo/:repoVersion/export/

• Notes
  • The passed authorization token must have administrative access to the repository in order to delete the export file

Useful HTTPie commands

• You can use httpie (installed on the server) to start/check export:

http HEAD $OCL_API_HOST/orgs/CIEL/sources/CIEL/55497986525c3173c33d2c4a/export/ Authorization:"Token $OCL_API_TOKEN" --print HBhb
http GET $OCL_API_HOST/orgs/CIEL/sources/CIEL/55497986525c3173c33d2c4a/export/ Authorization:"Token $OCL_API_TOKEN" --print HBhb