Status

Current state: [ UNDER DISCUSSION | ACCEPTED | REJECTED ]

Discussion thread: <link to mailing list DISCUSS thread>

JIRA: SAMZA-TBD

Released: 

Problem

Samza today supports RocksDB and MemDB as local data stores, which enables users to cache data for later usage during stream processing. However, the population of a data store is end user’s responsibility. This introduced complexity of maintaining local data stores, especially corner cases such as reload after consumers falling off. To avoid these issues, some people employed alternative solutions such as Voldemort, CouchBase, etc. As we look at the issue more closely, it appears generic enough to be addressed by data infrastructure.

Motivation

Proposed Changes

Public Interfaces

Implementation and Test Plan

Compatibility, Deprecation, and Migration Plan

Rejected Alternatives