You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Status

Current state: [ UNDER DISCUSSION ]

Discussion thread: <link to mailing list DISCUSS thread>

JIRA:  Unable to render Jira issues macro, execution error.

Released: 

Problem:

Samza today provides various api's like high level (fluent), low level (task) with configuration Samza-Yarn, Samza as a library and in coming future Samza will support In-memory system, that a user can produce/conusme from. With 0.13 release Samza provide a high level fluent api for chaning complex jobs, but still Samza users have to get their hands dirty with too much configuration details and dependency on external system(like kafka) to write integration tests. In order to alleviate this we intend to provide a fluent test api to users. This will help in setting-up quick tests against their system without depending on underlying incoming stream from a Kafka Topic. The goal of the proposed SEP we will provide a standardized framework to set-up continuous integration test for High Level Api, Low Level Api in any configuration of Samza with ease. Users won’t have to worry about the underlying Samza config details or even understand low level implementation. This abstraction will increase developer productivity and help them make their Samza system robust.

Motivation

Addition of this Continuous Integration Test Framework will alleviate:

  1. Lack of a standardized way to set up integration tests for users

  2. Lack of brevity in code and understanding low level config details of the Samza Api to set up samza systems

  3. Lack of a pluggable Test System with any Samza configuration

  4. Set of robust tests for current samza api

Assumptions 

  1. System depends on In Memory system to spin up in-memory system consumers and producers wherever applicable

  2. Initial system runs on SingleContainerGrouperFactory, that means these test are supposed to be ran on a single container environment

  3. In order to provide a testing api with least config management, basic configs are set up for users, but flexibility to add/override some custom config is also provided

Proposed Changes



Public Interfaces


Implementation and Test Plan


Compatibility, Deprecation, and Migration Plan

 

Rejected Alternatives

 

  • No labels