Page History

...

1. API server will have its own war file to deploy as a separate webapp from cloudstack engine, and can even be deployed to distributed machines to load balance user requests.
2. API server will have its own cache DB to store entity information that we want to present to end-user. Note that this cache DB is totally decoupled from CloudStack DB (which is shared by multiple management servers). Each api server can has its own cacheDB. This cache DB can have totally different schema design to represent a correct end-user targeted response view. The cache DB schema will closely map to our new cloudstack resource model plus row-level permission information that can acquired from pluggable ACL component.
3. With independent cache DB on each API server, API server becomes stateless, shutting down one api server will not impact other api server peers.
4. When API server starts, it will build up the cache DB by fetching fresh data from cloud-engine through REST api, and invoking ACL apis to fill in row-level permission information.
5. API server will subscribe to our event framework to get notified for each resource update to update its cacheDB.
6. During cacheDB buildup, we can also build inverted index using Lucene to support google-like search against it.
7. API server will provide a REST compliant api with the following REST pattern:

...

A pre-requisite to come up with a clean REST style API is to build CloudStack Resource Object Model. This object model should have the following information for each CloudStack resource, including those resources directly managed by CloudStack (like VM, volume, network, etc) and those indirectly related resources (like Domain, Account, etc). Note that NorthBound API and SouthBound API can share the same object model, controlled by different visibility flags.

1. Accessibilty of this resource entity, is it public to the end user or internal to CloudStack internal? For example, PhysicalNetwork should be a resource not public to end user, so API server should not expose a REST url with PhysicalNetwork as the <entity> in above URL pattern.
2. Each resource should be characterized by:
   • Attributes (including its visibility)
   • Relationships (characterizing various different relationships, like association, aggregation, composition, inheritance etc)
   • Supported Methods (including construction, delete, and operations, etc)

With this object model in place, we can have a generic and standard mechanism to

• Construct each Response class, instead of current ad-hoc way based on UI.
• Create cache DB schema or view based on resource response.
• Clearly identify supported methods for each resource.
• Provide a natural map of entity supported methods to REST url pattern. For example, POST create will easily map to the Constructor method of an entity, REST DELETE will map easily to the Delete method of an entity, etc.

Here is a simple Object Model snippet to briefly illustrate the idea: (In the diagram below, entities colored with light blue should serve as our first-level entities that can be exposed to end-user in our REST url pattern, while those entities colored with light brown should be internal to cloud-engine southbound API).
Image Added

Project Planning

Design and implementing this brand-new API server is not an easy task, and need coordinated work from other component, like cloud-engine, event notification, ACL, UI, internal service refactoring, etc. We need to take a phased approach to reach our final goal. Here are some subtasks I can identified so far and specified them in order here:

1. Define CloudStack Object Model for both API server and cloud-engine API.
2. Implement a new set of Response classes based on object model defined. We may need two set of Response classes, one for API server, one for Cloud-Engine API due to different granularity of information.
3. Create API server Cache DB view schema based on responses to be presented to the user.
4. Cloud-engine should provide REST api for api server to build the cache.
5. ACL component should provide APIs for api server to populate row-level permission information for each resource cached.
6. Cloud-engine should hook up with event framework to publish any resource update change.
7. API server needs to subscribe to event framework to be notified by each resource update change.
8. Implement API server cache build and update algorithm.
9. Building inverted index for API server cache db view using Lucene to support real keyword search.
10. Analyze all current Cmd classes and map to our standarized REST pattern, and implement them using JAX-RS implementation Apache CXF.