...
draw.io Diagram | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
On another note, ordering could still be guaranteed. It works in a similar manner to getMetadataAsync()
were it to call the client-given RecordCallback
once. So for ordering to happen, we will have to wait for a thread to finish processing its assigned segment of records first before any other records from other threads are returned. We are still processing using multiple threads at the same time but in different offset ranges. Let' s illustrate an example. For simplicity purposes, let parentThread
process offsets 1 - 50
, and childThread
process offsets 51 - 100
.
- For us to guarantee ordering,
childThread
's records cannot be returned untilparentThread
has finished processing theirs. So untilparentThread
finishes processing offsets1-50
, the results forchildThread
will have to be stored temporarily. - Each time a call for more metadata is made,
childThread
's results will be sent toparentThread
via callback. But they will not be sent until all offsets before it has been sent. - At some point, offsets
1-50
has been returned. In this case, we will returnchildThread
's records as well if anothermetadataForStore
method call was made. So basically, we are delaying the sending of records processed bychildThread
until the offsets before it has been received by the client.
This would work, but there might be some difficulties in implementing it.
Compatibility, Deprecation, and Migration Plan
...