CFP is here: http://events.linuxfoundation.org/events/apachecon-north-america/program/cfp

### Submission type: Presentation

### Audience Category: Developer

### Experience Level: Intermediate / Advanced

## Abstract:

Apache REEF (Sergiy Matusevych, Microsoft) - Resource managers like Apache YARN and Mesos have emerged as a critical layer in the cloud computing system stack, but the developer abstractions for leasing cluster resources and instantiating application logic are very low-level. We present Apache REEF, a powerful yet simple framework that helps developers of big data systems to retain fine-grained control over the cloud resources and address common problems of fault-tolerance, task scheduling and coordination, caching, interprocess communication, and bulk-data transfers. We will guide the developers through a simple REEF application and discuss current state of Apache REEF project and its place in the Hadoop ecosystem.

## Audience:

Apache REEF (Sergiy Matusevych, Microsoft) - This talk is for the developers of big data systems. By guiding the audience through a simple Apache REEF application, we give a low-level introduction to big data systems, and show how Apache REEF bridges the gap between traditional big data systems like Apache Spark, and low level API of resource managers like Apache YARN or Mesos. We demonstrate how Apache REEF provides the basic control flow for data-driven applications and gives the developer a powerful collection of general-purpose big data tools, like bulk data transfers, task management, and fault tolerance, without imposing any specific programming model.

## Linux/OSS gain:

Apache REEF (Sergiy Matusevych, Microsoft) - Recently graduated as a top level Apache project, REEF is already used in several large-scale production systems. Apache REEF fills an important niche in the Hadoop ecosystem, and opens Hadoop to the community of C#/.NET developers. It allows developers to mix Java and .NET tasks in one distributed job, and, to our knowledge, is currently the only framework that provides .NET API to Apache YARN and Mesos resource managers.

## Speaker Bio:

Sergiy is a research engineer at Microsoft Cloud and Information Services Lab, where he is building large scale distributed systems for big data and machine learning. He is a committer to the Apache REEF project. Prior to Microsoft, Sergiy worked as a data research engineer at Yahoo! Research, and tried his hand in building machine learning systems at several Silicon Valley startups. Sergiy is interested in machine learning, data stream processing, and high performance distributed systems. His previous speaking experience includes UAI, MLDS, and HadoopSummit conferences.