Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The Hadoop community believes that further development on Submarine can be done better as a separate project as discussed on the Hadoop email lists (here [submarine-dev@See: https://lists.apache.org/thread.html/70cd31946bdffaef6f83bfd32da031d2eb2710a263da27907914491a@%3Cyarn-dev.hadoop.apache.org]org%3E).

Proposal

Although Submarine was originally developed in Apache Hadoop, there are several forces that are encouraging it to move to a separate project:

...

Submarine’s development team seeks to foster the development and user communities. We feel that becoming a separate project will improve both communities by being smaller and more focused than Hadoop and bring tighter integration with various Apache projects and other open source projects that either doesn’t want to or can’t accept the large list of dependencies from Hadoop.

Core Developers

Hadoop Submarine is being primarily developed by Cloudera, NetEase, LinkedIn, JD, Dahua and , Ke.com., Facebook, Alibaba

Alignment

The ASF is a natural host for Submarine given that it is already the home of Hadoop, Spark, Hive, Arrow and other emerging distributed computing software projects. Submarine was designed to offer improved user experiences of deep-learning/machine-learning model training, serving, management which can be part of big data pipeline and leverages the power of Apache Spark, Apache Arrow, Apache Zeppelin, etc.

...

The potential PMC of the new project has extensive experience with Apache projects and includes [TODO number] Apache members and Incubator PMC members3 PMC members from Apache top level projects, 1 Apache member, and 1 Incubator PMC member. The Submarine PMC and the more experienced committers will be responsible for training the committers that are less familiar with the Apache Way.

...

The developers include employees from Cloudera, NeteaseNetEase, LinkedIn, JD, Dahua, Ke.com, Facebook, Alibaba TODO. Apache projects encourage an open and diverse meritocratic community and Submarine team is very motivated to increase the size and diversity of the development team.

...