Page History

...

CFP is here: http://events.linuxfoundation.org/events/apachecon-north-america/program/cfp

## Bio:

Sergiy is a research engineer at Microsoft Cloud and Information
Services Lab, where he is building large scale distributed systems for
big data and machine learning. He is a committer to the Apache REEF
project. Prior to Microsoft, Sergiy worked as a data research engineer
at Yahoo! Research, and tried his hand in building machine learning
systems at several Silicon Valley startups. Sergiy is interested in
machine learning, data stream processing, and high performance
distributed systems. His previous speaking experience includes high
profile conferences like UAI, MLDS, and HadoopSummit.

...

Resource Managers like Apache YARN and Mesos have emerged as a
critical layer in
the cloud computing system stack, but the developer
abstractions for
leasing cluster resources and instantiating
application logic are very
low-level. This flexibility comes at a high
cost in terms of developer
effort, as each application must repeatedly
tackle the same challenges
(e.g., like fault-tolerance, task scheduling and
coordination) , and
re-implement common mechanisms (e.g., caching, like caching and
bulk-data
transfers). In this talk we .

We present Apache REEF, a development
framework that provides a
control-plane for scheduling and
coordinating task-level (data-plane)
work on cluster resources
obtained from a Resource Manager. REEF
provides mechanisms that
facilitate resource re-use for data caching,
and state management
abstractions that greatly ease the development of
elastic data
processing work-flows on cloud platforms that support a
Resource
Manager service.

## Audience gain:

By guiding the audience engineers through a simple Apache REEF application, we
give a low-level introduction to big data systems, and show how Apache
REEF bridges the gap between traditional big data systems like Apache
Spark, and low level API of resource managers like Apache YARN or
Mesos. We demonstrate how Apache REEF povides the basic control flow
for data-driven applications and gives the developer a powerful
collection of general-purpose big data tools, like bulk data
transfers, task management, and fault tolerance, without imposing any
specific programming model. We also discuss the current state of the
Apache REEF project and its place in the Hadoop ecosystem.

...

Originated at Microsoft Cloud and Information Services Lab and
recently graduated as a top level Apache project, Apache REEF is yet
another evidence of Microsoft commitment to support open
source. Battle tested in multiple large-scale production systems,
Apache REEF fills an important niche in the Hadoop ecosystem, and
opens Hadoop to the community of C#/.NET developers.