Motivation

The purpose of this document is to outline the design of refactored HTTP (aka REST) API for AsterixDB.

The reason for HTTP API refactoring is threefold: 

• Currently we have various REST endpoints (/ddl, /query, /update, /aql, /query/service), which do similar things (parsing parameters out of the request parameters, assembling the response) but each endpoint does it in a slightly different manner. All of that should be refactored to use the common base endpoint. At the same time old endpoints could be kept for backwards-compatibility if that it really needed.

• Current Web interface does the server-side HTML generation and does not use REST calls whatsoever. It would be better to eat our own dog food here and switch to assembling the result on the client side using JavaScript. Especially since we have a potential GSoC project proposal to build such JS-based interface.

• As we are moving towards having more query languages under AsterixDB umbrella (AQL, SQL++, XQuery/JSONiq as a part of VXQuery) it would be nice to design a generic language-agnostic REST API, which later could be reused by VXQuery since it's also lacking proper API as of now.

Tools and existing APIs

The design is inspired by N1QL REST API (http://developer.couchbase.com/documentation/server/4.5/n1ql/n1ql-rest-api/executen1ql.html) since it is an example of well thought API in the similar system. I believe we don't need to be 100% compatible, although it would be nice to be able to reuse the same clients.

We might also consider using Swagger (http://swagger.io/) to describe the API. This library will allow users to seamlessly generate client SDKs in their favorite language, which is especially usefully since we don't provide drivers for any clients. Here is the complete set of features which Swagger will allow us to do:

• Describe API in developer-friendly way by creating yaml description.
• Validate correctness of the server-side implementation.
• Generate client-side SDK for various languages.
• Generate documentation for API.

REST Endpoints

Proposed API consists of 3 endpoints: Query, Status and Result. The latter two are needed only in a case of asynchronous data delivery, while the former is the main endpoint and serves as an entry point for all client requests.

Query Endpoint (/query/service)

The following compares various parameters & HTTP headers using in N1QL API and in current AsterixDB API and proposes parameters to be used in new API.

HTTP Request(POST) parameters:

HTTP Response format:

Type of the response - JSON document (even if format in request was CSV it will be serialized as one of JSON attributes)

Status Endpoint (/query/service/status)

This endpoint is supposed to be used only in the case when results are delivered asynchronously. The endpoint purpose is solely to inform about status of submitted query, and possibly include results of its execution

HTTP Request(GET) format:

http://localhost:19002/query/service/status/ID

Where ID is an identifier generated and returned by the /query/service endpoint.

HTTP Response parameters:

The response is a small subset of response of the /query/service endpoint

Examples:

1. Query which returns runtime error
   

   Request

   curl -v http://localhost:19002/query/service/status/27-2 -X GET

   Response

   < HTTP/1.1 500 Internal Server Error
   Content-Type: application/json
   {
     "status": "fatal",
     "error": {
       "code": 99,
       "msg": "Something happen during query execution",
       "stacktrace": "java.lang.RuntimeException: \n\r at java.lang.Thread.run(Thread.java:745)"
     }
   }

2. Query which is still executing
   

   Request

   curl -v http://localhost:19002/query/service/status/27-2 -X GET 

   Response

   < HTTP/1.1 200 OK
   Content-Type: application/json
   {
     "status": "running"
   } 

3. Query which successfully completes and return URI to its results. Reffer to Query endpoint Example 7 to see the original request.
   

   Request

   curl -v http://localhost:19002/query/service/status/27-2 -X GET

   Response

   < HTTP/1.1 200 OK
   Content-Type: application/json
   {
     "status": "success",
     "handle": "http://localhost:19002/query/service/result/27-2"
     "metrics": {
       "executionTime": 100ms,
       "resultCount": 10
     }
   }  

Result Endpoint (/query/service/result)

This endpoint is used to retrieve results of asynchronous query execution or synchronous results, when the client opted out from having results included into /query request (include-results=false).

Note that the Content-type of the result is set up appropriately to Accept in the initial request.

HTTP Request(GET) format:

http://localhost:19002/query/service/result/ID

Where ID is an identifier generated and returned by the /query/service endpoint (include-results=false) or by /query/service/status endpoint when the asynchronous result is computed.

HTTP Response format:

The response includes only the results with appropriate Content-Type header. 

Examples:

1. Synchronous query results. Refer to Query endpoint Example 6 to see the original request.
   

   Request

   curl -v http://localhost:19002/query/service/result/27-2 -X GET 

   Response

   < HTTP/1.1 200 OK
   Content-Type: application/x-adm
   {
     "id": 1,
     "screen_name": "BarackObama",
     "message_text": "Four more years"
   }
   {
     "id": 2,
     "screen_name": "ElonMusk",
     "message_text": "I Would Like to Die on Mars, Just Not on Impact"
   }

2. Asynchronous query results. Refer to Query endpoint Example 7 & Status endpoint Example 3 to see the original requests.
   

   Request

   curl -v http://localhost:19002/query/service/result/27-2 -X GET

   Response

   < HTTP/1.1 200 OK
   Content-Type: application/json
   [ {
     "id": 1,
     "screen_name": "BarackObama",
     "message_text": "Four more years"
   }, {
     "id": 2,
     "screen_name": "ElonMusk",
     "message_text": "I Would Like to Die on Mars, Just Not on Impact"
   } ]