Versions Compared

Old Version 47

changes.mady.by.user Bhavani Sudha

Saved on

compared with

New Version 48

changes.mady.by.user Vinoth Chandar

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Roadmap

Below is a depiction of what's to come and how its sequenced

draw.io DiagrambordertrueviewerToolbartruefitWindowfalsediagramNameroadmapsimpleViewerfalsewidthdiagramWidth1382revision4This is a rough roadmap (non exhaustive list) of what's to come in each of the areas for Hudi.Under construction (smile), early 2021 unveiling

Writing data & Indexing 

  • Improving indexing speed for time-ordered keys/small updates
    • leverage parquet record indexes,
    • serving bloom filters/ranges from timeline server/consolidate metadata
    • Indexing the log file, moving closer to scalable 1-min ingests
  • Improving indexing speed for uuid-keys/large update spreads
    • global/hash based index to faster point-in-time lookup
  • Incrementalize & standardize all metadata operations e.g cleaning based on timeline metadata
  • Auto tuning 
    • Auto tune bloom filter entries based on records
    • Partitioning based on historical workload trend
    • Determination of compression ratio

...