Page History

...

In order to improve concurrent filter query performance, CarbonData supports a "Search Mode" to do query scheduling and query execution without using Spark Scheduler and Executor. In one of our test, it can achieve half latency comparing to using Spark for filter query on indexed column, from 1 second to 500 milliseconds. 

Other Important Improvements

• Improved EXPLAIN command output to show: whether the query is rewritten to use Pre-Aggregate Table or Materialized View, which index DataMap is hit, and how many blocklets are pruned.
• Added log for performance tuning information, including driver side parsing and optimizer time, bock distribution info, carbon file IO read time, number of blocklet scanned, result filling time to spark, etc
• Support compaction and loading in parallel.
• Support separating visible and invisible segment metadata into two files and show them separately in SHOW SEGMENTS command

• Support global sort option on partition table

• Reduced object generation in global sorted table

• Optimization on DESC command to show partition value and location for partitioned table

Please find the detailed JIRA list: https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12341005&styleName=Html&projectId=12320220&Create=Create&atl_token=A5KQ-2QAV-T4JA-FDED%7C72f8d21d9927bf947fc8c0dfb7f69263d4048efb%7Clout

...