A page to capture thoughts and notes about the Tuscany SCA distributed runtime. Feel free to help in developing the ideas here. I've put the source for the diagrams on this page in my sandbox here
While the nature of a distributed runtime implies that more than one runtime of more than one type will be involved in a running SCA application, it seems sensible to work with distributing one type of runtime (java) before branching out.
Distributed SCA Runtimes - Notes
The assembly model specification 2 deals briefly with distributed runtimes in its discussion of SCA Domains
"An SCA Domain represents a complete runtime configuration, potentially distributed over a series of interconnected runtime nodes."
The assembly spec, however, is not prescriptive about how an SCA Domain should be mapped and supported across multiple runtime nodes. Here I believe the term runtime node (or just node) is used to describe a process running an SCA runtime in which components can be run, e.g. the Java or C++ runtimes that Tuscany is developing.
Furthermore, the SCA specifications do not talk about the form of repository in which the metadata of the SCA Domain is held - although clearly the metadata must live somewhere. The Domain repository could take a variety of forms, from a simple shared set of directories to a sophisticated distributed database system. SCA runtimes should be able to interact with a variety of Domain repositories.
Motivation
Some use cases that provide motivation for distributing a domain
- Represent the widely distributed nature of a typical SOA so that SCA presents a cross enterprise description of assembled components
- Support policy matching where components require particular resources and hence particular, and separate, nodes
- HA/Load balancing/Performance scenarios where a single component appears on multiple nodes
- Load balancing/Performance scenarios where domain is spread across multiple nodes (same as 1 & 2 I believe)
- Dynamic wiring/Registry based service location, i.e. the SCA binding is called upon to automatically locate services based on registry entries.(overlaps with all of the above)
Terminology
See http://cwiki.apache.org/confluence/display/TUSCANYWIKI/Terminology
Scoping The Distribution Problem
There are many existing technologies that deal with managing compute nodes and job scheduling. So it's probably safe to start by ignoring the issue of how the system picks processors on which runtime nodes will run. So the runtime management of the nodes themselves is out of scope.
There are also many technologies that provide scalable, robust and/or high performance service hosting solutions. So we can also ignore the issue of how component instances are actually constructed as the runtime representation of components deployed to a runtime. For example if a JVM clustering solution is chosen to implement a node then we assume that local method calls within that cluster will be handled by the clustering technology and no special action is required. If higher level clustering technology is in operation then intergration with the runtime is required. In this case, where each node in the cluster runs part of the domain a component can be mapped to multiple nodes, the most natural integration point is the SCA binding which must interact with the clustering technology in order locate target component services.
So the initial area of consideration is how the components of a domain are associated with runtime nodes (2).
Cardinality
In the non-distributed case a single runtime node loads all contributions and runs all components.
In the distributed case, A Domain may span many nodes.
Each component must be associated (through specific configuration or some matching algorithm) with one or more nodes. If the same component appears in more than one node, the runtime is responsible for deciding how messages are routed to the correct node.
There is no restriction on the number of component instances a node can create, or indeed how these component instances are run. For example, all component instances could run in a single VM or be distributed across a number of VM's to provide improved performance or failover for example.
Some questions have been raised about cardinality
Should load balancing or HA type scenarios be able to be described in the topology by allowing components to be assinged to more than one node?
Answer: Yes. However, this may well be best handled by a "layered runtime" approach, where a whole cluster of nodes is presented to the rest of the distributed runtime as a single node.
Should a runtime be able to run more than one domain?
Answer: Yes. Multiple Domains can run on the same runtime. It is up to the runtime implementation to ensure that appropriate partitioning is achieved, since SCA Domains are intended to be isolated (for example, a reference in one domain cannot directly reference a service in another domain through it SCA component & service names).
Scenario - Distributed Calculator
Scenario - Web Application Cluster
A more specific scenario where a distributed domain is used to support an application within a web application configured as a cluster.
Managing The Distributed Domain
Th logical view of how the different parts of the solution communicate is.
Messages - the application messages that flow between configured components. Messages will flow over bindings described excplicitly in the assembly model or across the default binding used when no explicit binding is specified.
Configuration - In the disitrubted domain configuration is shared across the nodes with which the domain is associated. This includes information about, contributed resources, running components and their endpoints and domain configuration items such as base URLs.
Events - as the domain runs interesting events will occur, for example, a node fails and is restarted meaning that a set of endpoints change.
Components Of The Solution
Based on the calculator scenario can imagine the following.
Note that the assumption here is that there is a central process that represents the node and a proxy in each node that provides a window on the domain from that node/JVM.
APIs
SCADomainFactory
SCADomain
- public void start() throws DomainException;
- public void stop() throws DomainException;
- public String getURI();
- public void addContribution(String uri, URL url) throws DomainException;
- public void removeContribution(String uri) throws DomainException;
- public void addToDomainLevelComposite(QName compositeQName) throws DomainException;
- public void removeFromDomainLevelComposite(QName compositeQName) throws DomainException;
- public void addDeploymentComposite(ContributionURI, CompositeXML) throws DomainException;
- public void startComposite(QName qname) throws DomainException;
- public void stopComposite(QName qname) throws DomainException;
- public <B, R extends CallableReference<B>> R cast(B target) throws IllegalArgumentException;
- public <B> B getService(Class<B> businessInterface, String serviceName);
- public <B> ServiceReference<B> getServiceReference(Class<B> businessInterface, String referenceName);
SCANodeFactory
SCANode
- public String getURI();
- public SCADomain getDomain();
- public void addToDomainLevelComposite(QName compositeQName) throws DomainException;
- public void removeFromDomainLevelComposite(QName compositeQName) throws DomainException;
- public void startComposite(QName composite) throws NodeException;
- public void stopComposite(QName composite) throws NodeException;
- public void start() throws NodeException;
- public void stop() throws NodeException;
- public void destroy() throws NodeException;
SPIs
NodeEvents (node to domain)
- public String registerNode(String nodeURI, String nodeURL);
- public String removeNode(String nodeURI);
- public void registerContribution(String nodeURI, String contributionURI, String contributionURL);
- public void unregisterContribution(String contributionURI);
- public String registerServiceEndpoint(String domainUri, String nodeUri, String serviceName, String bindingName, String URL);
- public String removeServiceEndpoint(String domainUri, String nodeUri, String serviceName, String bindingName);
- public String findServiceEndpoint(String domainUri, String serviceName, String bindingName);
NodeManagement (domain to node)
- public String getURI();
- public void addContribution(String contributionURI, String contributionURL);
- public void deployComposite(String compositeName);
- public void start();
- public void stop();
Interactions
Action |
Domain |
Domain Proxy |
Node |
Notes |
---|---|---|---|---|
|
|
|
|
|
Starting Node standalone |
|
|
SCANodeFactory nodeFactory = SCANodeFactory.newInstance(); |
|
|
|
|
SCANode node = nodeFactory.createSCANode(null, null); |
use default node URL and don't connect to a domain |
Starting Domain |
SCADomainFactory domainFactory = SCADomainFactory.newInstance(); |
|
|
|
|
SCADomain node = domainFactory.createSCADomain(null) |
|
|
use default domain URL on this machine |
Starting Node to connect to domain |
|
|
|
|
Starting Domain proxy standalone |
|
|
|
|
Add contribution to domain |
|
|
|
|
Add contribution to node |
|
|
|
|
Remove Contribution from domain |
|
|
|
|
Remove Contribution from node |
|
|
|
|
Update Contribution in domain |
|
|
|
|
Update Contribution in node |
|
|
|
|
Start domain |
|
|
|
|
Stop domain |
|
|
|
|
Start node |
|
|
|
|
Stop node |
|
|
|
|
start contribution at node |
|
|
|
|
stop contribution at node |
|
|
|
|
Start composite at domain |
|
|
|
|
Stop composite at domain |
|
|
|
|
start composite at node |
|
|
|
|
stop composite at node |
|
|
|
|
Get service from domain |
|
|
|
|
Get service from domain proxy |
|
|
|
|
Get service from node |
|
|
|
|