Versions Compared

Old Version 3

changes.mady.by.user Hernan Cunico

Saved on

compared with

New Version 4

changes.mady.by.user Hernan Cunico

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 4.0

...

This document will begin the task of :

  • enumerating clustering requirements in Geronimo's tiers
  • abstracting out commonality from these
  • cataloguing software available to us that may be used to implement them
  • suggesting a clustering architecture

A 'Cluster' is an architecture that achieves scalability and high availability through the arrangement of multiple smaller, cheaper, less reliable, resources, rather than single large, expensive, extremely reliable ones.

...

We will enumerate and examine these usecases.

Back to Top

Web

URL: http://java.sun.com/products/servlet/download.html#specsImage Removed
Interest Group: Jules Gosnell, Jan Bartel, Jeff Genender, David Jencks,...

Clustering in the web-tier has two points of implementation :

The HTTP Load Balancer:

  • our solution should work with any load-balancer
  • we can expect the LB to support some form of affinity (sticky or persistant sessions).

The HttpSession:

  • large numbers of potentially large objects (more data in the tier than can comfortably be held in one node) - needs partitioned cache
  • typically frequently written
  • typically frequently read
  • typically a single consumer/client
  • transactionless
  • only one copy of an HttpSession may be 'active' at any one time
  • multi-threaded
  • pessimistic
  • suggested impl - WADI

N.B.: Some other solutions allow the clustering of more than just the HttpSession. The spec only requires that data stored in the HttpSession is distributable.

Dev Threads
WADI and Network Partitions: http://www.mail-archive.com/dev%40geronimo.apache.org/msg15855.htmlImage Removed
WADI/AS merger - http://www.mail-archive.com/dev@geronimo.apache.org/msg14749.htmlImage Removed

Back to Top

EJB

URL: http://java.sun.com/products/ejb/docs.html#specsImage Removed
Interest group: David Blevins, Gianny Damour

...

The Client/Proxy (equiv to Web Load-balancer):

  • cluster-aware proxies needed
  • can same proxies work with both OpenEJB and IIOP protocols ?

The Server:

MDB

  • stateless

SLSB

  • stateless

SFSB

  • large numbers of potentially large objects (more data in the tier than can comfortably be held in one node) - needs partitioned cache
  • transactional
  • typically frequently written
  • typically a single consumer/client
  • typically frequently read
  • single threaded
  • pessimistic
  • suggested impl - WADI

Entity

  • potentially multiple consumers
  • mapped to shared, persistant store
  • transactional
  • read/write frequency variable, pluggable impls required (e.g. RO Beans etc..)
  • distributed db backed cache needed
  • since mapped to db, does not need partitioned cache, data can just be unloaded
  • distributed caching of entity beans: http://www.mail-archive.com/dev%40geronimo.apache.org/msg15697.html

...

  • suggested impl - ActiveSpace

Dev Threads:
client stubs and load-balancing: http://www.mail-archive.com/dev@geronimo.apache.org/msg13533.htmlImage Removed
AC in client stub: http://www.mail-archive.com/dev@geronimo.apache.org/msg14756.htmlImage Removed
Entity invalidation...

...

Interest Group: Rajith Attapattu http://java.sun.com/products/jndi/1.2/javadoc/Image Removed

Apache Directory - http://directory.apache.org/Image Removed (may use this?)

  • small amounts of small objects
  • typically seldom written
  • typically frequently read
  • typically multiple consumers/clients
  • multi-threaded
  • transactionless
  • prime candidate for straight forward 1->all replication
  • suggested impl - ActiveSpace

Dev Threads:
ActiveSpace in JNDI: http://www.mail-archive.com/dev@geronimo.apache.org/msg14743.htmlImage Removed
WADI/AS merger - http://www.mail-archive.com/dev@geronimo.apache.org/msg14749.htmlImage Removed

Back to Top

JMS

http://java.sun.com/products/jms/docs.htmlImage Removed

  • impl - ActiveMQ

The ActiveMQ team will happily look after the clustering of JMS. For more information see the current Clustering Support in ActiveMQ

Dev Threads:
AMQ Clustering: http://www.mail-archive.com/dev%40geronimo.apache.org/msg15717.htmlImage Removed

Deployment

http://jcp.org/en/jsr/detail?id=088Image Removed

  • potentially large objects (perhaps we could just send references)
  • union of tier content smaller than single node capacity
  • seldom written (undeployed)
  • frequently read (invoked)
  • transactionless
  • no uniqueness constraint ? (what should we do about creating heterogeneous deployments?)
  • prime candidate for straight forward 1->all replication
  • suggested impl - ActiveSpace

suggestion - 'deployments sets' - a groups of nodes that share/implement a homogeneous deployment. See "Homogeneous vs Heterogeneous Deployments" section.

...

suggestion - could replication be done synchronously and serially, so that as the app is deployed on each node it can be sanity checked before the distribution continues to the next node ?

WADI,AC,AS & Deployment: http://www.mail-archive.com/dev@geronimo.apache.org/msg15214.htmlImage Removed

Back to Top

Management/Monitoring

http://jcp.org/en/jsr/detail?id=077Image Removed

  • still little discussion here
  • may need to be centralised
  • history - perhaps selected statistics snapshots should be dropped
    into JMS every few secs by each server, then stashed in an rrdb ?
    (WHEN, WHERE, WHAT, VALUE)

Back to Top

POJO

JCache - http://jcp.org/en/jsr/detail?id=107Image Removed

  • all of the above - whatever is available to Geronimo should be available to application-space pojos.
  • JCache, and related technologies...
  • suggested impls - ActiveSpace/WADI

Back to Top

DB

any takers ?

...

Network Partitions: check out 'Totem' thread on dev - not yet archived...
Application Session: http://www.mail-archive.com/dev@geronimo.apache.org/msg12072.htmlImage Removed
clustering shopping list: http://www.mail-archive.com/dev@geronimo.apache.org/msg10561.htmlImage Removed

Back to Top

Web Services

...

Back to Top

Clustering Substrate

  • suggested impl - ActiveCluster on top of various protocols

Back to Top

Auto-wiring

  • clients need to autolocate nodes (ejb-client->jndi-port, http-load-balancer->http=port etc...)...
  • nodes on same box need to negotiate ownership of per-host resources (e.g. ports)

Back to Top

Homogeneous vs Heterogeneous deployments

  • homogeneous cluster is one set of nodes - the universal set
  • heterogeneous cluster is one set (the universal set), with subsets defining internal scopes

Perhaps ActiveCluster could be extended to allow subclusters via the
same Cluster connection, or we could use multiple cluster
connections, or an AMQ MessageGroups...?

...

Suggested Building Blocks

ActiveMQ

URL: http://incubator.apache.org/activemq/Image Removed
Interest Group: James Strachan, Hiram Chirino, ...

...

Back to Top

ActiveCluster

URL: http://activecluster.codehaus.org/Image Removed
Interest Group: James Strachan, Hiram Chirino, ...

...

Back to Top

ActiveSpace

URL: http://activespace.codehaus.org/Image Removed
Interest Group: James Strachan, Hiram Chirino, ...

Provides two abstractions, a JCache style Cache (replicated and/or transactional) and a JavaSpaces style Space, for distributed computing.

Back to Top

WADI

URL: http://wadi.codehaus.org/Image Removed
Interest Group: Jules Gosnell,...

...

Back to Top

Tomcat Clustering

URL: http://tomcat.apache.org/tomcat-5.0-doc/cluster-howto.htmlImage Removed
Interest Group: Jeff Genender, ...

...

Back to Top

EVS4J/TOTEM

URL: http://www.bway.net/~lichtner/evs4j.htmlImage Removed
Interest Group: Guglielmo Lichtner

Extended Virtual Synchrony for Java.
Potentially very useful in cases where we need fast, strictly ordered 1->all messaging.
Might be integrated via ActiveCluster or ActiveMQ

Introduction: http://www.mail-archive.com/dev%40geronimo.apache.org/msg15644.htmlImage Removed
Totem and ActiveCluster: http://www.mail-archive.com/dev%40geronimo.apache.org/msg15695.htmlImage Removed
Infiniband: http://www.mail-archive.com/dev%40geronimo.apache.org/msg15743.htmlImage Removed

Back to Top

Further Discussion

Back to Top

Further Reading

http://portal.acm.org/citation.cfm?id=326136Image Removedhttp://scholar.google.com/url?sa=U&q=http://historical.ncstrl.org/tr/ps/cornellcs/TR99-1726.psImage Removedhttp://www.jgroups.org/javagroupsnew/docs/index.htmlImage Removedhttp://citeseer.csail.mit.edu/amir95totem.htmlImage Removedhttp://docs.codehaus.org/display/WADI/LibraryImage Removedhttp://citeseer.ist.psu.edu/32449.htmlImage Removed

"Fault Tolerance in Distributed Systems"
Pankaj Jalote, 1994
Chapter 7, Section 5, "Degree of Replication"