ID | IEP-55 |
Author | |
Sponsor | |
Created | |
Status |
DRAFT...
...
...
Motivation
The current approach for Ignite configuration was formed a long time ago (originally inherited from GridGain during the project donation) and has several disadvantages that manifested themselves over time as project gained more usage across different users. There are several orthogonal aspects that need to be addressed.
...
Dynamic cache configuration persistence depends on whether the cache is persistent or not. A memory-only dynamic cache will not be recreated on a cluster restart (even if there are other persistent caches), while persistent caches are preserved.
Description
The purpose of this IEP is to provide a revisited unified approach to configuring an Ignite cluster, which addresses the issues stated above by achieving the following goals:
...
Other configuration formats are open for discussion throughout the IEP finalization.
Keys order restriction
Both HOCON and JSON formats don't preserve keys order when parsed, it's a part of the specification. So it's proposed to have extra syntax for elements that require strict order:
Code Block |
---|
firstline | 1 |
---|
title | HOCON keys order |
---|
|
// Default syntax.
root.namedList {
key1 { value = val1 },
key2 { value = val2 }
}
// Specific syntax for when order must be preserved.
root.namedList = [
{ elementName = key1, value = val1 },
{ elementName = key2, value = val2 }
] |
Global and local properties
...
Code Block |
---|
language | text |
---|
title | Global tree |
---|
|
cluster.caches = {
partitioned = {
mode = PARTITIONED,
backups = 2
},
replicated = {
mode = REPLICATED,
atomicity = TRANSACTIONAL
}
} |
Code Block |
---|
language | text |
---|
title | Local tree |
---|
|
node.rebalance.threads = 4 |
For a full list of node-local and cluster-wide properties refer to the Appendix A of this document. It is kept up-to-date and contains all standard properties.
Configuration lifecycle
Configuration files are not exposed directly for the user for edit. Instead, a user is provided with an interface which allows to initialize a node and a cluster during the cluster setup and tooling which allows to modify to change configuration properties. Initial configuration can be provided in either way supported by HOCON.
...
Code Block |
---|
language | text |
---|
title | Change rebalance threads |
---|
|
ignitectl -node consistent_id node.rebalance.threads=4 |
Similarly, the tool can read property or the whole subtree value, export properties to a file or even update the cluster configuration based on a file.
Code Block |
---|
language | text |
---|
title | More examples |
---|
|
#read the property value
ignitectl cluster.baseilne.auto-adjust.timeout
#write values from the config file
ignitectl -w custom.properties
#update local properties on on all nodes in current topology
ignitectl -node * node.rebalance.threads=4 |
Appendix A - Full list of configuration properties
Last updated: May 11 2022
Code Block |
---|
language | text |
---|
title | Node-local properties |
---|
|
clientConnector.port
clientConnector.portRange
clientConnector.connectTimeout
clientConnector.idleTimeout
compute.threadPoolSize
compute.threadPoolStopTimeoutMillis
network.port
network.portRange
network.shutdownQuietPeriod
network.shutdownTimeout
network.inbound.soBacklog
network.inbound.soReuseAddr
network.inbound.soKeepAlive
network.inbound.soLinger
network.inbound.tcpNoDelay
network.outbound.soKeepAlive
network.outbound.soLinger
network.outbound.tcpNoDelay
network.membership.membershipSyncInterval
network.membership.failurePingInterval
network.membership.scaleCube.membershipSuspicionMultiplier
network.membership.scaleCube.failurePingRequestMembers
network.membership.scaleCube.gossipInterval
network.nodeFinder.type
network.nodeFinder.netClusterNodes
rest.port
rest.portRange |
Code Block |
---|
language | text |
---|
title | Cluster-wide properties |
---|
|
pageMemory.defaultRegion.pageSize
pageMemory.defaultRegion.persistent
pageMemory.defaultRegion.initSize
pageMemory.defaultRegion.maxSize
pageMemory.defaultRegion.memoryAllocator.type
pageMemory.defaultRegion.evictionMode
pageMemory.defaultRegion.replacementMode
pageMemory.defaultRegion.evictionThreshold
pageMemory.defaultRegion.emptyPagesPoolSize
pageMemory.defaultRegion.checkpointPageBufSize
pageMemory.defaultRegion.lazyMemoryAllocation
pageMemory.regions.<name>.* # same properties as in default region
rocksDb.defaultRegion.size
rocksDb.defaultRegion.writeBufferSize
rocksDb.defaultRegion.cache
rocksDb.defaultRegion.numShardBits
rocksDb.regions.<name>.* # same properties as in default region |
Risks and Assumptions
New configuration approach will require more effort from existing users during migration. Additionally, it will require a thorough configuration revisit and clear separation of global and local properties.
Discussion Links
TBD
Reference Links
https://github.com/lightbend/config
Tickets
Open Tickets
Jira |
---|
server | ASF JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
maximumIssues | 20 |
---|
jqlQuery | project = IGNITE AND (labels = iep-55 OR "Epic Link" = IGNITE-14904) AND status NOT IN (resolved, closed) |
---|
serverId | 5aa69414-a9e9-3523-82ec-879b028fb15b |
---|
|
Closed Tickets
Jira |
---|
server | ASF JIRA |
---|
columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
---|
maximumIssues | 20 |
---|
jqlQuery | project = IGNITE AND (labels = iep-55 OR "Epic Link" = IGNITE-14904) AND status IN (resolved, closed) |
---|
serverId | 5aa69414-a9e9-3523-82ec-879b028fb15b |
---|
|
TBD