Powered By Spark

This page tracks the users of Spark. Feel free to add yourself to this list (you will need a wiki user account) and explain how you use Spark. Please add a short description (up to three bullet points) and a link to your organization or project.

Companies & Organizations

• UC Berkeley AMPLab - Big data research lab that initially launched Spark
• Adatao, Inc. - Pervasive Data Science in the Enterprise
  • Team of ex-Googlers & Yahoos with large-scale infrastructure experience (including both flavors of MapReduce at Google & Yahoo) & PhD's in ML/Data Mining
  • Determined that Spark, among the many alternatives, answered the right problem statements with the right design
• GraphFlow, Inc.
• Autodesk
• Falkonry
• Amrita Center for Cyber Security Systems and Networks
• Exabeam
• Premise
• Digby
• Celtra
• Nokia Solutions and Networks
• UC Santa Cruz
• Istanbul Sehir University
• Ooyala, Inc. - Powering personalized video experiences across all screens
  • See our blog post on how we use Spark for Fast Queries
  • See our presentation on Cassandra, Spark, and Shark
• MediaCrossing - Digital Media Trading Experts in the New York and Boston areas
  • We are using Spark as a drop-in replacement for Map/Reduce to get the right answer to our query in a much shorter amount of time.

Software Projects

• Shark - Hive and SQL on top of Spark
• MLbase - Machine Learning project on top of Spark
• Apache Mesos - Cluster management system that supports running Spark
• Tachyon - In memory storage system that supports running Spark
• BigR - Native R (and other front-ends) for Big-Data-Science/Machine-Learning with open API on top of Spark+Hadoop (soon to be open sourced)