...
Generator:
generation of timestamped events (bids, persons, auctions) correlated between each other
NexmarkLauncher:
creates sources that use the generator
queries pipelines launching, monitoring
Output metrics:
Each query includes ParDos to update metrics
execution time, processing event rate, number of results, but also invalid auctions/bids, …
Modes:
Batch mode: test data is finite and uses a BoundedSource
Streaming mode: test data is finite but uses an UnboundedSource to trigger streaming mode in runners
Queries pseudo code
Query 0 (not part of original NexMark): Pass-through.
...