...

Note that the assumption here is that there is a central process that represents the node and a proxy in each node that provides a window on the domain from that node/JVM.

...

APIs

SCADomainFactory

SCADomain

• public void start() throws DomainException;
• public void stop() throws DomainException;
• public String getURI();
• public void addContribution(String uri, URL url) throws DomainException;
• public void removeContribution(String uri) throws DomainException;
• public void addToDomainLevelComposite(QName compositeQName) throws DomainException;
• public void removeFromDomainLevelComposite(QName compositeQName) throws DomainException;
• public void addDeploymentComposite(ContributionURI, CompositeXML) throws DomainException;
• public void startComposite(QName qname) throws DomainException;
• public void stopComposite(QName qname) throws DomainException;
• public <B, R extends CallableReference<B>> R cast(B target) throws IllegalArgumentException;
• public <B> B getService(Class<B> businessInterface, String serviceName);
• public <B> ServiceReference<B> getServiceReference(Class<B> businessInterface, String referenceName);

SCANodeFactory

SCANode

• public String getURI();
• public SCADomain getDomain();
• public void addToDomainLevelComposite(QName compositeQName) throws DomainException;
• public void removeFromDomainLevelComposite(QName compositeQName) throws DomainException;
• public void startComposite(QName composite) throws NodeException;
• public void stopComposite(QName composite) throws NodeException;
• public void start() throws NodeException;
• public void stop() throws NodeException;

Interactions

Contribution Added To Domain

Old Nots

Node

• start(nodeUri)
• stop()
• joinDomain(domainUri)
• domainNodeConfigurationChange(domainUri)

Provides management functions for a Node. The node may be started and stopped. Once started it can join a domain and accept configuration changes. Configuration changes will be in the form of contribution URLs

ServiceDiscovery

• findServiceEndpoint(domainUri, serviceName) url
• registerServiceEndpoint(domainUri, serviceName, url)

In the distributed environment nodes must communicate to exchange information about which services they each expose. This interface allows nodes to register the service endpoints they provide and find service endpoints that other nodes provide.

DomainNode

The node may have joined one or more domains. This set of interfaces allows each domain node (the part of the domain that belongs to a node) to be managed.

DomainNode

• createDomainNode(domainUri, nodeUri)
• startDomainNode(domainUri, nodeUri)
• stopDomainNode(domainUri, nodeUri)

The node domain may be started and stopped.

BaseUriMap

• setBaseUri(domainUri, nodeUri, protocol, uri)
• getBaseUri(domainUri, nodeUri, protocol) uri

An interface for getting the base uri configuration into the node domain and out again.

ComponentMap

• addComponent(domainUri, nodeUri, componentName)
• removeComponent(domainUri, nodeUri, componentName)
• getComponents(domainUri, nodeUri) list of component names

Allows a distributed domain to tell each node domain which components it will be responsible for. This is likely to be a subset of the full set of components that a domain defined.

ContributionManager

• addContribution(domainUri, nodeUri, contributionUri)
• removeContribution(domainUri, nodeUri, contributionUri)

The mechanism by which contributions are added to and removed from a node domain.

ComponentManager - a version is already defined in host embedded

• startComponent(domainUri, nodeUri, componentName)
• stopComponent(domainUri, nodeUri, componentName)

To start and stop individual named components. All the components at node domain can be started/stopped by starting/stopping the node domain itself.

Distributed Domain

• registerNode(domainUri, nodeUri, nodeManagementUrl)
• getDomainNodeConfiguration(domainUri, nodeUri) url

Allows nodes to join a domain and retrieve their current configuration. This interface should also cover configuration updates but I son;t know what the granularity of these udated should be or how they should be represented.

Event (I expect there is some specified interface we can use here)

• logEvent(domainUri, nodeUri, event)
• getEventLog(domainUri, nodeUri,)
• getEventLog(domainUri)

Walkthrough

1. Running a node

• Run a node exe giving it a node uri and a domain to join.
• The node exe will embed and start a Tuscany runtime.
• Node service is exposed
• Node will discovery where the distributed domain is running
  • in a file based scenario configuration is available locally
  • discovery can be hardcoded if required.
• Create and start a domain node

2. Running the distributed domain

• Start the distributed domain exe giving
  • Note the distributed domain may only exist in configuration files on disc and in this case no separate exe is required
• Gather together the domain configuration
  • base uris
  • Added contributions
  • Components added to nodes
• Nodes will join the domain as they are started
• Provide domain node configuration to a node on request
• If the configuration changes notify each (affected) node

3. Node initial configuration

• Requests configuration for this domain node
• Configuration is supplied in the form of
  • base uris
  • contributions to load
  • components to activate
• Contributions are loaded
  • Gives rise to endpoints being registered with the distributed domain

4. Starting a domain node

• Domain node is activated
  • currently gives rise to all domain components starting

5. Starting a component

• Start a named component

6. Stopping a component

• Stop a named component

7. Stopping a domain node

• Domain node is stopped
  • all running components are stopped

8. Updating node configuration

• Distributed domain notifies all (affected) nodes
• Node retrieves domain node configuration (updates)
  • New/Updated/Removed controbution
  • Added/Remove components
  • Currently incremental domain updated are not fully supported so will have to go with wholesale reconfiguration
• Domain node is stopped
• Contributions are reprocessed
• Domain is restarted

9. Choosing a component instance

Assigning components to nodes defines the endpoint for a components services. The distributed domain uses this information to create default bindings for the cross node wires. If a component is assigned to multiple nodes then the runtime is responsible for selecting the appropriate node based on, scope, conversational status of target component and also non specified goals such as load balancing

10. Events and Stats

• The hierarchy of components in the distributed domain
• The components running on a node
• Events/logs for the distributed domain or for a node

11. Node failure

• A failed node is restarted and reconfigures itself from the distributed domain
  • and inflight requests are lost
  • any ongoing conversations are lost unless they have been persisted by the runtime
  • endpoints are re-registered when node restarts
• A failed node can be restarted in a different place
  • Base uri configuration must be adjusted to take account of new location.
  • endpoints are re-registered when node restarts

12. Distributed domain failure

• Nodes remain running in isolation.
  • periodically trying to rediscover new distributed domain
• Restart distributed domain
• nodes should eventally rediscover it

SCA Binding

The SCABinding is the default binding used within an SCA assembly. In the runtime in a single VM case it implies local connections. In the distributed runtime case it hides all of the complexity of ensuring that nodes wired between runtimes are able to communicate.

When a message oriented binding is used here we benefit from the abstract nature of the endpoints, I.e queues can be created given runtimeId/ServiceID and messages can be targetted at these queues without knowledge of where the message consumers are physically.

Whene a point to point protocol is used a physical endpoint is required. So a registry of endpoints to SCA bound service is required to allow the SCA binding to find the appropriate target. This registry can either be static, i.e. derived from the base urls given in the domain topology configuration, or dynamic in nature, i.e. set up at runtime.

Within the same domain/runtime multiple technologies may be required to implement the SCA binding as messages pass between different runtime node implementations.

Modelling The Distributed Domain

Using information from the SCA Assembly specification and the implied requirements of a distribute runtime we can determine what data is required to configure and control the distributed SCADomain.

SCADomain
  Name (DomainA)
  BaseURI
  Domain Level Composite
    Component (ComponentA)
      implementation
        composite
      Service
      Reference
    Installed Contributions
    Initial Package
    Contribution (file system, jar, zip etc)
      URI (ContributionA)
      /META-INF/
        sca-contribution.xml
          deployable (composite QName)
          import (namespace, location)
          export (namespace)
        sca-contribution-generated.xml
          deployable (composite QName)
          import (namespace, location)
          export (namespace)
        deployables
          *.composite
      *.composite
        URI
        Component (ComponentA)
          Service
          Reference
      Other Resources
        URI
      Dependent Contributions
        Contribution snapshot
      Deployment-time Composites
        *.composite

Over and above the contributed information we need to associate components with runtime nodes.

Runtime
  name (runtimeA)
  Node
    name (nodeeA)
    DomainA
      scheme http://localhost:8080/acbd
      scheme https://localhost:442/abcd
      ComponentA

We know how SCDL is used to represent the application composites. We can view the runtime node configuration as a new set of components, interfaces, services and references. In SCA terms we can consider that each node implements a system composite that provides the service interfaces required to manage the node, for example.

<composite xmlns="http://www.osoa.org/xmlns/sca/1.0"
           name="nodeA">
    <component name="ComponentRegistry">
        <implementation.java class="org.apache.tuscany.sca.distributed.node.impl.DefaultComponentRegistry"/>
    </component>
</composite>

Having this meand that we can expose out local component registry using any bindings that Tuscany supports. Imagine that out component registry has an interface that allows out to

getComponentNode
setComponentNode
etc.

Then we might choose to initialise the registry with the follwoing type of information.

<runtime>
    <node name="nodeA">
        <schema name="http" baseURL="http://localhost:80" />
        <schema name="https" baseURL="https://localhost:443" />
        <component name="CalculatorServiceComponent" />
    </node>
    <node name="nodeB">
        <schema name="http" baseURL="http://localhost:81"/>
        <schema name="https" baseURL="https://localhost:444" />
        <component name="AddServiceComponent"/>
    </node>
    <node name="nodeC">
        <schema name="http" baseURL="http://localhost:81"/>
        <schema name="https" baseURL="https://localhost:444" />
        <component name="SubtractServiceComponent"/>
    </node>
</runtime>

Of course we can read this configuration locally form a file, have it delivered via a service interface or retrieve it via a reference.

To Do

SCA Binding

Currently the code uses JMS to implement the default remote SCA binding. The remote SCA binding is used when the system finds that two components that are wired together locally are deployed to separate Nodes. As an alternative it would be good to support web services here also and have this fit in with the new SCA binding mechanism that Simon Nash has been working on.

To make a web services SCA binding work we need an EndpointLookup interface so that components out there in the distributed domain can locate other components that they are wired to.

Node

Currently each node runs in isolation and starts a local SCA domain configured from .topology and .composite files. Now implement Node interfaces so that this information can be provided remotely and so that the node can expose remotely accessible management interfaces, for example.

The domain management interface Ant has recently been added that may help us shape this. Also Sebastien's work to allow local domains to be modified more dynamically should help make this work.

Domain Node

Provide the link between the node and the domain node in order to deliver configuration updates, control messages and events

Distributed Domain

Provide some centralized control by implementing the distributed domain interfaces

WebApps

It qould be useful to have some simple query application in the tone of what is currently in distribution/webapp

References

...

• public void destroy() throws NodeException;

SPIs

NodeEvents (node to domain)

• public String registerNode(String nodeURI, String nodeURL);
• public String removeNode(String nodeURI);
• public void registerContribution(String nodeURI, String contributionURI, String contributionURL);
• public void unregisterContribution(String contributionURI);
• public String registerServiceEndpoint(String domainUri, String nodeUri, String serviceName, String bindingName, String URL);
• public String removeServiceEndpoint(String domainUri, String nodeUri, String serviceName, String bindingName);
• public String findServiceEndpoint(String domainUri, String serviceName, String bindingName);

NodeManagement (domain to node)

• public String getURI();
• public void addContribution(String contributionURI, String contributionURL);
• public void deployComposite(String compositeName);
• public void start();
• public void stop();

Interactions

Load Balancing