You are viewing an old version of this page. View the current version.
Compare with Current
View Page History
« Previous
Version 4
Next »
One frequent usage pattern for Ignite is bulk data loading. Users need to be able to load data to Ignite from external sources as fast as possible. Ignite is not optimized for this use case at the moment, as bulk data loading process goes through the same code paths as normal cache updates. We need to improve bulk data loading performance.
All proposed changes can be split in two groups - infrastructure improvements and index improvements. Note that some proposals are in conflict with each other so careful evaluation is a must.
Infrastructure improvements:
- Disable WAL for some caches when doing bulk data load
- Bypass GridCacheMapEntry
- Add data to new pages rather to existing pages to minimize free-list overhead
- Perform cache scan through data pages rather than primary index pages
Index improvements:
- Bypass H2 engine when updating index
- Fill index pages in batches
- Build index using bottom-top approach
- Build index from multiple threads
- Optimize index comparisons for date/time data types
key |
summary |
type |
created |
updated |
due |
assignee |
reporter |
priority |
status |
resolution |
JQL and issue key arguments for this macro require at least one Jira application link to be configured
|