THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
Status
Current state: [ DISCUSS ACCEPTED ]
Discussion thread: DISCUSS
...
| Gliffy Diagram | ||||
|---|---|---|---|---|
|
Data Injection:
In the context of this Test framework an input stream means a stream that Samza job may consume from and an output stream means a stream that Samza job can produce to. The initial source of data stream is always supposed to be bounded since this is a test, and that may originate as one of the following sources listed below.
...
| StreamAssert | StateAssert |
|---|---|
| contains | contains |
| containsInAnyOrder | satisfies |
| inWindowPane | |
| satifies |
Data Types & Partitions:
The framework will provide complete flexibility for usage of primitive and derived data types. Serdes are required for local kafka stream and file stream but in memory streams dont require any Serde configuration. Serdes will be also required if users want to maintain State. Test framework will provide api's for initialization of input streams (both single and multi-partition), and also data validation on single partition and multi-partition of the bounded streams
...
Shown below is a targeted test matrix plan to test various components of Samza in p2.
n = number of threads
x = number of partitions
m = number of containers
| Samza API | Concurrency | Partitions | Container | Expected Result | ||
|---|---|---|---|---|---|---|
StreamTask | task.max.concurrency = 1 job.container.thread.pool.size = n | 1 <= p <= nx | 1 / nm | in order processing | ||
task.max.concurrency > 1 job.container.thread.pool.size = n | 1 <= p <= nx | 1 / nm | out of order processing | |||
AsyncStreamTask | task.max.concurrency = 1 | 1 <= p <= nx | 1 / nm | in order processing | ||
| task.max.concurrency = n | 1 <= p <= nx | 1 / nm | out of order processing | |||
| Windowable Task | N/A | 1 <= p <= nx | 1 / nm | expecting processing n messages for messages window of time t | ||
| Initiable Task | N/A | 1 <= p <= nx | 1 / nm | stateful testing (assertions on kv store) | ||
| Closable Task | N/A | 1 <= p <= nx | 1 / nm | verify closing the client? | ||
Map / Flatmap / Filter / Partition By / Merge / SendTo / Sink / Join / Window
| task.max.concurrency = 1 job.container.thread.pool.size = n | 1 <= p <= nx | 1 / | nm | in order processing | |
task.max.concurrency > 1 job.container.thread.pool.size = n | out of order processing | |||||
...