Note taker :  <@username>

Agenda

Status Updates 

Blockers, Progress towards next releases, things needing immediate attention, JIRA cleanup


  • All of the below, will be shared in DISCUSS thread on dev and users MLs
  • Release process :
    • Major version x.y → x.y+1 every 3 months (feature driven)
    • Minor version release x.y.z → x.y.z+1 every month or so (time based)
    • major release features need to be guarded by flags, on minor versions.
    • always releasing from master, bump up major version once all committed features are working
    • Some cases, we may have to just make a patch releases on top. But only extenuating circumstances
  • Major Release planning :
    • PMC/Committers come up with an initial list sourced based on user asks, support issues
    • List is shared with the community, for feedback. community can suggest new items, re-prioritization
    • Contributors are welcome to commit more features/asks, (with due process)
  • Next Minor version 0.6.1 (with stuff that did not make it to 0.6.0..) 
    • Flink/Writer common refactoring for Flink
    • Small file handling support w/o caching
    • Spark3 Support
    • Remaining bootstrap items
    • Completing bulk_insertV2 (sort mode, de-dup etc)
  • 0.7.0 with major new features 
    • RFC-15: metadata, range index (w/ spark support), bloom index (eliminate file listing, query pruning, improve bloom index perf)
    • RFC-08: Record Index (to solve global index scalability/perf)
    • RFC-18/19: Clustering/Insert overwrite
    • Spark 3 based datasource rewrite (structured streaming sink/source, DELETE/MERGE)
    • Incremental Query on logs (Hive, Spark) 
    • Parallel writing support
    • Redesign of marker files for S3 
    • Stretch: ORC, PrestoSQL Support

Pull Request Review 

Unblock reviews, Resolve sticky points

  • All reviews assigned

RFC Reviews

Go over major open ended items

  • RFC-15: updated with implementation approach. 

Open Floor 

new ideas, feedback, suggestions.


Major Followups





  • No labels