This page describes ideas, thoughts, and planning around Apache Ignite 3.0 release.
One of the biggest challenges with the current version of Ignite is related to how it works with data schemas. Here are the main issues:
To solve this, we should switch to the "schema-first" approach, which means that the schema is defined for a cache/table before its creation. The proposed changes are:
BinaryObject
API is replaced with Record
API (the naming can be discussed, of course). Here, a record is essentially a binary tuple that represents a set of fields stored in a cache entry or a table row. Similar to current binary objects, records have a "field" method to extract individual fields. A record can also be deserialized to any subset of fields.Another pain point is the fact that all the configuration in Ignite is static. Almost any updates to the configuration require cluster restart. It is also not obvious which parameters can be updated and which cannot because all of them are located within the same set of beans – there is no separation.
The proposal is to go through all the parameters and create a clear separation between:
Once this separation is done, we will need to update internal logic to allow changes for the dynamic parameters – such changes must be possible without any node restarts. The set of dynamic parameters can be minimal in the first version, as long as there is a way to convert static parameters to dynamic going forward.
In addition, the current Spring XML format seems to be outdated – it's not even recommended for Spring itself. Not to mention that we have to add the whole Spring dependency solely for configuration conversion. We suggest using the LightBend Config library based on the HOCON format: https://github.com/lightbend/config. This format is designed specifically for configuration purposes, is compatible with JSON, Java Properties, and allows merges. The latter is very useful for dynamic configuration updates.
Currently, query API is attached to the cache API. I.e., to execute a query, one needs to create a cache first. This is incompatible with SQL-only use cases where all the tables are created via DDL. In this use case, executing a SQL query requires that the user creates a dummy cache without no data and uses this cache as an API gateway. This is very counterintuitive. Instead, we should have a designated API for SQL, which would be located on the top level (e.g. ignite.sql().executeQuery(..)
).