ID | IEP-104 |
Author | |
Sponsor | |
Created | 26/05/2023 |
Status | |
Motivation
...
- Checks metadata (mappings, binary_meta, caches - can check inside Ignite, not reading files), prepare updates if any.
- Polls the Queue, transforms ReadSegment data to Iterator<CdcEvent>, pushes them to CdcConsumer.
- If CdcConsumer#onEvents returns true:
- Persists CdcConsumerState.
- Write
RealtimeCdcRecord
record to WAL with the WALPointer.
- Optimization: transform segment buffers to CdcEvents in background (to reduce the buffer usage). CdcConsumer should be async then?
Try restart Ignite's internal CDC processswitch to the realtime mode:
- User sends the command to switch modes
- Ignite does initialization - CdcWorker, buffer
- Writes
TryStartRealtimeCdcRecord
into WAL and rollover current segment(since this record realtime cdc becomes active again). - Ignite monitors the CDC directory, awaits while segment with the record cleaned - it means ignite-cdc.sh reach the record and stops capturing the data.
- If buffer is not overflowed in this moment - Ignite enables CDCConsumer and starts sending the records
- Otherwise, ordinal stop is invoked (with writing StopRealtimeCdc record)
...