Page History

This page is meant to start as a conceptual design for a 2nd generation API for ESME. The idea is to learn from the discussions around and use of the original API.

h1. API Design

{toc}

h2. Streams vs. Resources

In the design below, all parts of ESME are modeled as resources, in keeping with a RESTful approach. For things like message streams, this is not an optimal way to model these entities. I'd like to maintain the ability to interface with the full ESME API in a RESTful manner, while also encouraging the use of more optimal interface approaches when this makes sense.

The existing entities called out as resources below that should be available in some manner as streams are:
* Messages (user-specific) api/users/USERID/messages
* Messages api/messages
* Tags (api/tags) 
* Conversations (api/conversations) 
* Pools (api/pools)
* Searches ??

References -
Dev mailing list thread - [http://www.mail-archive.com/esme-dev@incubator.apache.org/msg00976.html]

h2. Methods, Resources, and Descriptions

*Bold* means the resource and method is implemented in the current /api2/ endpoint.

||Resource||Method||Description/Payload schema/Response schema||Streaming?||
|api2/session|*GET*,*POST*,*DELETE*|Post parameter: token|
|api2/users|*GET*|
|api2/users/{USERID}|GET|
|api2/user/messages|*GET*,*POST*|Post parameters: message, via (opt), pool (opt), realm (opt), metadata (opt), tags (opt), replyto (opt)|Yes (partially implemented)| 
|api2/user/tags/{TAG}|*GET*| | Yes |
|api2/user/followees|*GET*|
|api2/user/followers|*GET*,*POST*| Post parameter: userId |
|api2/user/followees/{USERID}|*DELETE*| 
|api2/user/tracks|*GET*,*POST*| Post parameter: track (regex) |
|api2/user/tracks/{TRACKID}|GET,*DELETE*|
|api2/user/tracks/{TRACKID}/messages|GET| |Yes|
|api2/user/actions|*GET*,*POST*| Post parameter: name, test, action |
|api2/user/actions/{ACTIONID}|GET,*PUT*,*DELETE*| Put parameter: enabled (boolean) |
|api2/messages/{MESSAGEID}|GET|
|api2/messages|GET,POST| |Yes|
|api2/conversations/{CONVERSATIONID}|*GET*|
|api2/conversations/{CONVERSATIONID}/messages|GET,POST| |Yes|
|api2/pools|*GET*,*POST*|
|api2/pools/{POOLID}|GET,DELETE|
|api2/pools/{POOLID}/users|GET,*POST*| Post parameters: realm, userId, permission |
|api2/pools/{POOLID}/users/{USERID}|DELETE|
|api2/pools/{POOLID}/messages|GET,POST| |Yes|

One point to note is that some HTTP clients do not currently support the "PUT" or "DELETE" methods, so these may have to be simulated through POST methods with an extra parameter. I think that because of the close mapping to resource verbs, is worth using these methods in the specification and defining the simulation method for the entire API separately.

h2. Streams

There are a lot of ways we can model streams and I'm very interested in input here, as this is not an area that I am familiar with (though I hope to learn quickly from some of the experts on the project). Options for interfacing to streams that I have seen:

* XMPP - [http://xmpp.org/]
* AMQP - [http://jira.amqp.org/confluence/display/AMQP/Advanced+Message+Queuing+Protocol]
* HTTP
** Polling (bad)
** Comet/long-polling - Bayeux - [http://svn.cometd.com/trunk/bayeux/bayeux.html]
** Reverse HTTP - [http://www.reversehttp.net/]
** PubSubHubBub? - possibly via Reverse HTTP - [http://www.reversehttp.net/demos/endpoint.html]

h3. ESME HTTP Comet/long-polling implementation for streams

In general, for any resource URL that indicates streaming is implemented above, we will implement an HTTP-based streaming interface. This interface might better be referred to as a "delta" interface. The URL will behave as follows (we use api2/user/messages as an example):

||URL||Request||Query Parameters||Behavior||
|api2/user/messages|GET| |Returns any messages not already read by the client|
|api2/user/messages|GET|timeout=20|If new messages not already read by the client exist, these are returned. If no new messages exist, the request will stall for a number of seconds as defined in the timeout= parameter. If a new message arrives during this time, the request will immediately resolve, returning the new message.
|api2/user/messages|GET|history=10|Returns the last 10 messages in the timeline|

Please note that this interface is not yet implemented on any of the api2 URLs. Once it is implemented where planned for a URL, the "Yes" above in the "Streaming?" column will be bolded.

{gliffy:name=ESME delta message HTTP API|space=ESME|page=API 2.0 - Design|pageid=2329310|align=left|size=L}

h3. Streams/Message-queues in ESME and in general

{gliffy:name=ESME message queues|space=ESME|page=API 2.0 - Design|pageid=2329310|align=left|size=L}

h2. Resource/Object/Stream Hierarchy
 
The above is based on a rough object hierarchy as follows: 

* ESME API instance (api/) 
** Sessions (api/sessions) 
** Messages posted by logged in user (api/user/messages) (1)
** Users followed by logged in user (api/user/followees) 
** Users following logged in user (api/user/followers) 
** Trackers belonging to logged in user (api/user/tracks) 
** Actions belonging to logged in user (api/user/actions) 
** Messages (api/messages) (1)
** Tags (api/tags) (1)
** Conversations (api/conversations) (1)
** Pools (api/pools) (1)
*** ?
** Searches ?? (1)
** Trends ??

(1) Stream interface should be available and use should be encouraged

Each of these bullets represents a set of objects. The resource representing an individual object lives at api/objects/OBJECTID. For example, api/sessions/SESSIONID. As much as is reasonable, one would expect to be able to GET (read), POST (create), PUT (update/amend), or DELETE (delete) any individual member of each of these object sets. Going through each of these objects to ask what it would mean to create, read, update, or delete that object may reveal holes in the existing API, some of which I have filled in above.

h2. Formats

h3. Request formats

*Format specification*

If there is a request body, format should be specified using the Content-Type HTTP header.

*Formats to be supported*

* XML ?
* JSON ?
* Multi-part Form-encoded ?
* Form-encoded

h3. Response formats

*Format specification*

Format could be specified using the HTTP Accept header - [http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html]

Another option (though not as robust) would be to append the format to the resource request url. For example /api/users/USERID.json

*Formats to be supported*

* XML ?
* JSON ?
* ...

h2. Authorization

Currently the ESME REST API uses tokens as the authorization mechanism. A token is used to establish a session and then the session is used to persist the authorization of the API client across the length of the session.

There are a couple of problems with this, though we don't have a better approach at the moment:
# Sessions are not natively supported in a lot of API programming environments, especially environments that do not have a persistent data-store available to the application.
# The current API design appears encourage that the token sent to establish the session be sent in the clear over an unencrypted connection.

h2. Points for discussion, resolution, further work

# Is the use of HTTP sessions necessary? Is it desirable?
# Request signing methods?
# Payload and response schemas must be defined
# Should API contain admin functions?
# Webhooks ([http://blog.webhooks.org/])
#* ESME has webhooks as part of its actions framework, but we may want to document their existence as part of the API, and possibly improve the functionality if there are use cases ([http://incubator.apache.org/esme/actions.html])
# What is a conversation?
# Authorization approach (see above)
# What is our streaming approach?