Page History

...

• i.e., if we have same some channel state on upstream, and channel state on downstream corresponds to it (as ensured by step 4); then it’s enough to sort them by channel id ([src_id : dst_id]); this will ensure order of records
• there is a (technical?) problem with the proposed solution to match MBRsmulti-buffer records: we need to alternate load channel and process network operations which can be tricky
• another solution would be to have temporary "virtual" channels to process data from "imported" channels

...

The task then must filter out irrelevant records by keys (after deserialization). For non-MBRs regular (single-buffer) records it’s trivial.

For MBRsmulti-buffer record, if MBR it doesn’t belong to this subtask, it will never receive remaining buffers and therefore can’t deserialize and filter it out.

...

1. Send restored records from upstream to all downstreams that could need it (not according to key); we can use the same logic as in distribution for that
2. Use sequence numbers: if we expect next part of MBR a multi-buffer record on this channel, but receive a buffer with a different SN; then discard this MBRstored buffer
3. Write key in the beginning of each buffer (need to ensure the key itself isn’t splitted)

...