...

Release plan(Around 3 months for one release)

Road map plan:

1.0.x:

• Support 2.1 integration in CarbonData
• Remove kettle, support new data load solution
• Support data update and delete SQL in Spark 1.6

...

• Support MV(Materialized View), Bloom Filter (in production features)
• Support CarbonData engine for improving concurrent visit and point query.
• Ecosystem integration
• Support alter add column in carbon file format
• Supports multiple character separators in csv file during data loading
  
• Compaction support for segments created with range_sort and global_sort
  
• Support DDLs to operate on Driver Cache (Get cache size, clear cache)
  
• Support building datamaps  and data load in parallel to reduce the overall time taken
  
• Summary of loaded and bad records data after data loading
  

1.6.x:

• Support storing of carbon Min-Max indexes in external system
• MV DataMap Enhancements and Stabilisation
• Query performance enhancements
• Deeper Presto integration and stabilisation
• UDF and UDAF support in Pre-aggregate tables
• Support Read from hive

2.0.x:

• Support Write into hive
• Load performance improvements
• TPCDS [Query, load] performance improvements
• Carbon Advisor for auto suggestion of ideal table schema including MV, index, sort col, range col, compression ...
• Delete and update support in CarbonData SDK
• Support C engine reader for CarbonData SDK
• ES based datamap management
• Support Spark DataSource API V2
• Support CarbonData metadata management using DB or other external OLTP system
• Support MV on Streaming tables, partition tables, Time Series
• Support MV creation from another MV

2.1.x:

• Presto read support for complex columns
• Make GeoID visible to the user
• Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.
• Implement delete and update feature in carbondata SDK.
• Support array<string> with SI
• Support IndexServer with Presto Engine
• Implementing a new Reindex command to repair the missing SI Segments
• Support Change Column Comment
• Support Local dictionary for presto complex datatypes
• Block Pruning for geospatial polygon expression
• Improve concurrent query performance
• Support global sort for Secondary index table
• Filter reordering
• Geospatial index algorithm improvement and UDFs enhancement
• CarbonData Trash support
• Support Writing Flink Stage data into Hdfs file system
• Support MERGE INTO SQL Command
• Support Complex DataType when Save DataFrame
• Adding global sort support for SI segments data files merge operation.

2.2.x:

• Support Add, Drop and rename column support for the complex column
• Spark-3.1 support
• Secondary Index Support for Presto
• CDC Performance improvement
• Local sort Partition Load and Compaction improvement
• Geo Spatial Query enhancements
• Improve table status and metadata writing

2.3.x:

• Support spatial index creation using data frame
•  Introduce Streamer tool for Carbondata
• Upgrade prestosql to 333 version
• Multi-level complex schema support
• Support for Dynamic Partition Pruning

Pages Link

Committers

Releases

CarbonData Performance Reports

Apache CarbonData Performance Benchmark(0.1.0)

...

Use cases and shared articles

...