Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Capturing from the buffer (background threadtransaction threads)

  1. In wal-sync-thread (the only reader of mmap WAL), under the lock that synchronizes preparing ReadSegment and rolling the WAL segment, to guarantee there are no changes in the underlying buffer.
  2. Offers a deep copy of flushing ReadSegments to the CdcWorker.
  3. CdcWorker checks remaining capacity and the buffer size
  4. If the size fits the capacity then store the offered buffer data into the Queue. 
  5. Otherwise: 
    1. remove from the queue tail segments to free space for the offered buffer
    2. store the head of the offered buffer as nextHead (WALPointer)
    3. It captures data from the segment buffers Queue while nextHead is not reached.
    4. Switch to the archiveMode.

Body loop

  1. bufferMode:
    1. polls queuethe Queue, transforms ReadSegment data to Iterator<CdcEvent>, pushes them to CdcConsumer.
    2. Optimization: transform segment buffers to CdcEvents in background (to reduce the buffer usage). CdcConsumer should be async then?
  2. archiveMode:
    1. similar to CdcMain - await archived segments
    2. submits the read WALRecords to the CdcConsumer.
    3. for every segment/record checks a condition to switch to the bufferMode:
      1. check the loaded WALPointer after initialization
      2. OR while nextHead is not reached
  3. In both modes it persists CdcConsumerState. Policy for committing the progress is the same - : by WAL segment.

Discussion Links

...