Flume is extensible and designed to be able to deliver data to many data storage and management systems. At its core it is designed to deliver data reliably to Hadoop's HDFS. It also has a plugin interface that allows contributers to add different sources and different sinks for their data. Here is a list of different systems Flume can or will deliver data to (NOTE: these are not supported by Cloudera):
- Cassandra Sink
- Elastic Search Sink
- Voldemort
- AMQP via RabbitMQ
- Hive (in progress) :: Video Hadoop World 2010: Scale in Collecting and Querying Log Data in Near Real-time - Anurag Phadke, Mozilla
- HBase :: Video Hadoop World 2010: Search Analytics with Flume and HBase - Otis Gospodnetic, Sematext International. Further Contributions from Alex Baranua and Dani Rayan.
- MongoDb (in progress)
- JRuby Plugins Chris Howe, Infochimps
- FlumeBase :: Streaming SQL queries for Flume. Aaron Kimball