API Requirements

We need to expose APIs for configuring, commanding and monitoring replication agents.

Configuration API

• Should allow CRUD operations for agent configs
• Configs need to have a JCR representation (either the default one or a new one) in order to support the following workflow: configure all agents on a author instance and replicate configuration to all publish instances.

Command API

• Should allow Replicate/Import Package/Export Package
• Commands might be issued to multiple agents at once (but this not a strong requirement I think)

Monitoring API

• Should allow inspection to internal queues of replication agents
• Should allow inspection of commands history

API endpoints 

Configuration API

• Create config  - POST /system/replication/config
• Read config - GET /system/replication/config/publish
• Update config - PUT /system/replication/config/publish
• Delete config - DELETE /system/replication/config/publish

Command API

For command and monitoring APIs we can have two kind of APIs. We do not need to implement both variants, and I do not think there is a strict requirement for choosing one variant or another.

The flattened APIs can take as parameters multiple agents and the granular APIs can take as parameters just on

Commands API - flattened

Replicate tree - POST /system/replication/all/replicate

Import package - POST /system/replication/all/import

Export package - POST /system/replication/all/export

Commands API - granular

Replicate tree - POST /system/replication/agent/{agentName}/replicate

Import package - POST /system/replication/agent/{agentName}/import

Export package - POST /system/replication/agent/{agentName}/export

Monitoring API

Monitoring APIs - flattened

Replicate tree - GET /system/replication/all/replicate/history?agent={agentName}

Import package - GET /system/replication/all/import/history?agent={agentName}

Export package - GET /system/replication/all/export/history?agent={agentName}

Internal queues  - GET /system/replication/all/queues?agent={agentName}&queue={queueName}

• Lists information about the packages in a queue

Monitoring APIs - granular

Replicate tree - GET /system/replication/agent/{agentName}/replicate/history

Import package - GET /system/replication/agent/{agentName}/import/history

Export package - GET /system/replication/agent/{agentName}/export/history

Internal queues  - GET /system/replication/agent/{agentName}/queues/{queueName}

• Lists information about the packages in a queue

API Implementation 

Custom servlets vs Standard servlets

There are two main to implement the APIs: use custom servlets or use standard servlets. 

• Standard servlets 
  • have the advantage that requests are stored in content and security comes for free
  • have the disadvantage that for commands API an additional call must be made to find out the status of the command
  • config apis fit very well the content profile
• Custom servlets
  • have the advantage of being able to give a status in the response after a command is placed
  • have the disadvantage of having to implement custom security checks inside the servlet.

Conclusion:

• Configuration API should be implemented using SlingPostServlet. 
• Command API can be implemented using SlingPostServlet if we can live with the asyncronous status check. It is important to have in mind that the replication commands are asynchronous by default as there might be some queues that are used the different parts of a replication request. Hence finding out if a replication request was completely finished will require inspecting the history most of the times.
• Monitoring API should be implemented with custom servlets at least for internal queues as they display live info which would be tedious to sync in the repo

Flatten vs granular 

The commands and monitoring APIs can be implemented using either a flat or a granular approach (or both)

• Flatten design
  • It permits sending requests to multiple agents which might be desirable at least for replicate requests
  • It is easier to implement as it requires no hierarchy of resources
• Granular design
  • It is resource oriented
  • One needs to implement either a ResourceProvider or a JCR syncronization for agents and queues in order to represent the hierarchy. There is no consensus on which is best.

 Conclusion: Granular design is desirable if we can get some agreement on how to represent the dynamic hierarchy.

Sample payloads

Replicate to multiple agents

code 

POST /system/replication/all/replicate

Request

{

"action" :  "add",

"paths" : [ "/content/tree1", "/content/tree2" ],

"agents" : [ "publish1", "publish2" ]

}

Response

{

"statusList" : [ { "agent" : "publish1", "status" : "queued" },

{ "agent" : "publish2", "status" : "error" } ]

"id" : "ds54eaw543rft4"

}

code

Import to multiple agents

code

POST /system/replication/all/import

Request

{

"action" :  "import",

"paths" : [ "/content/tree1", "/content/tree2" ],

"agents" : [ "publish1", "publish2" ],

"package": "packagestream"

}

-----------------------------separator

Content-Disposition: form-data; name="packagestream"; filename="vltpackage.zip"

Content-Type: application/octetstream

(binary data)

-----------------------------separator

Response

{

"statusList" : [ { "agent" : "publish1", "status" : "queued" },

{ "agent" : "publish2", "status" : "error" } ],

"id": "dfqw43t3g"

}

code