Versions Compared

Old Version 9

changes.mady.by.user Vinoth Chandar

Saved on

compared with

New Version Current

changes.mady.by.user Shaofeng Li

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Current log format is depicted here. Its a simple, yet flexible format that consists of - version, type (avro data, deletes, command), header, optional content, footers. Headers and footers allow us to encode various metadata like commit instant time, schema for processing that log block. On DFS implementations that support append, there will be multiple of these log blocks present in a single log file version. However, on cloud/object stores without append support 1 log file version contains 1 log block. 

...