• Created by Unknown User (starchmd), last updated on Nov 25, 2014 1 minute read

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

What is Streaming OODT?

Streaming OODT (S-OODT) is an attempt to bring technologies that support big-cluster processing and stream processing into the OODT processing system. This will enable OODT to tackle bigger data sets and new data paradigms moving into the future. The core technologies handle: stream management, cluster management, processing, file system efficiency and backward-compatibility (running standard OODT within a managed cluster).   These technologies include:

  • Apache Kafka (Stream management)
  • Apache Mesos (Cluster management)
  • Apache Spark and Spark Streaming (Processing)
  • Tachyon and Hadoop HDFS (File system efficiency)

For detailed design documentation please refer to:

What is required for Streaming OODT?

Streaming OODT is designed for use on clusters of machines.  It requires at least one head node and at least one slave node.  The head node runs the main servers, and the slaves run the client services including processing slaves and HDFS slaves.  In general the cluster should have moderate local disk storage and above normal RAM on each slave node. Currently S-OODT has only been tested on linux hosts.

In order to install and run S-OODT the following items are reuired to be configured on your cluster before you install.

 

Required software:

  • Apache Maven
  • Java
  • GNU Compiler Tools (gcc, g++)
  • SVN

Required environment variables:

  • M2_HOME
  • JAVA_HOME

Using Streaming OODT

 

  • No labels