...

Motivation

Currently, Apache Ignite is being shipped as a single monolith binary with lots of excess modules and libs, the majority of which are not required in most use case scenarios.
Also, current upgrade process is too tied to binary delivery, requiring users to copy the working directory, additional libs, and other configurations between version upgrades, which is rather error-prone.

This IEP targets to solve both problems the following way:

...

:

2. apache-ignite-core with essential-only files to run Apache Ignite
3. apache-ignite-extras with non-essential parts like benchmarks, examples, etc.
Enhance ignite.sh script  & other control scripts to untie it from IGNITE_HOME and use Maven as the main additional lib management system.
4. Update command-line utility to provide the minimal required module management
5. Provide command-line extensibility which is in line with the new distribution model

Final target – recreate binary delivery in a way, that will allow a user to upgrade Apache Ignite instance with only downloading a new binary and executing the same ignite .sh command from for the new version.

Description

Binary Delivery Revamp

1. Redesign binary assembly to prepare at least 2 archives:

   • apache-ignite-core with essential-only files to run Apache Ignite: bin, config, libs (without optional)

   • apache-ignite-extras with all other additional files: benchmarks, docs, examples, platform

2. Correspondingly update RPB/DEB packages to mirror the new binary layout.
   Considering packages, modularisation can be further enhanced to have separate packages for each additional part of the delivery (apache-ignite-benchmarks, apache-ignite-examples, etc.)

Additional Libs Management System

Replace shipping of optional libs via including Maven-powered dependency management.

1. Completely remove optional libs from any delivery.
2. Update ignite.sh to support specifying additional libs names and use Maven to download target libs with dependencies.

Configurations Enhancements

1. The binary delivery is narrowed down to a single ignite command-line utility which is the essential toolchain for installing ignite modules, managing classpath, and node lifecycle.

2. Correspondingly update RPB/DEB packages to include the ignite CLI utility

Additional Libs Management System

Replace the shipping of all Ignite libs with Maven-powered dependency management:

1. The utility is equipped with a set of commands to install arbitrary Maven artifacts using Maven coordinates (e.g. mvn:org.apache.ignite:ignite-core:2.9.0)
2. The utility has a set of built-in aliases for Maven artifacts to simplify default modules installation (e.g. server->[mvn:org.apache.ignite:ignite-core:2.9.0, mvn:org.apache.ignite:ignite-indexing:2.9.0, mvn:org.apache.ignite:ignite-spring:2.9.0]
3. The utility should be able to support additional Maven repositories, as well as external resource locations (SFTP, HTTP(s), etc)

Configurations Enhancements

For well-defined and predictable library management, the utility has a number of indirections when resolving work, configuration, and library storage paths. The resolution path depends on whether ignite was configured locally or globally for a machine/container.

Versioning & Installation

Ignite installation from a user perspective narrows down to a single package containing the command-line tool install via one of the package managers (apt-get, rpm, homebrew) or a manual download. Once the command-line tool is installed, further modules management is performed from the tool.

The utility version directly matches the server and client library versions which this utility will manage, i.e. ignite v.${version} will use org.apache.ignite:ignite-core:${version}, org.apache.ignite:ignite-spring.${version} and so on.

In order to update Ignite, a user updates the single ignite package, which will later take care of updating installed modules. A user can easily switch between ignite versions using regular update-alternatives (or similar) tools.

Library and work dir resolution

To differentiate between local and global configuration mode, the utility first attempts to read the .ignitecfg file in the user local directory ${user.home}. If this file exists, then the library and work directory paths are defined in there. If the file does not exist, the utility attempts to read the .ignitecfg file in a predefined system folder (e.g. it may be /etc/ignite for Linux systems, /Library/Ignite for MacOS, Program Files\Ignite for Windows). If this file exists, then the library and work directory paths are defined there. If none of the files exist, then the library and work directories default to ${user.home}/ignite-bin and ${user.home}/ignite-work.

file = if ${user.home}/.ignitecfg exists
           ${user.home}/.igniterc
       else if ${system.default}/.ignitecfg exists
           ${system.default}/.igniterc
       else
           None

if file is None {
    libsDir = ${user.home}/ignite-bin
    workDir = ${user.home}/ignite-work
} else {
    libsDir, workDir = read from file
}

Modules management

Modules and external libraries can be installed and uninstalled using ignite install or ignite uninstall commands. The command accepts a built-in alias for one or many Maven coordinates, maven coordinates, or a URI for external resource. The set of installed modules is written to a config file ignite.modules located in the ${workDir} folder, resolved previously (the configuration file is updated whenever a module is installed or uninstalled). Upon module installation, ignite resolves corresponding artifacts and its dependencies from Maven and places them to the ${libsDir}/${version} folder. Additionally, ignite pre-builds and saves a classpath formed by all installed modules in a separate file for faster node start-up.

A default directory structure of installing Ignite 3.0.1 and then upgrading to Ignite 3.0.2 will look as follows:

/
+
+---+Library/Ignite                 ignite
|    +                                 +
|    +---+3.0.1                        |
|    |    +                            |
|    |    +---+ignite-3.0.1            |
|    |                                 |
|    +---+3.0.2                        |
|         +                            |
|         +---+ignite-3.0.2 <----------+
|
+---+Users/John
     +
     +---+ignite-bin
     |    +
     |    +---+3.0.1
     |    |    +
     |    |    +---+ignite-core-3.0.1.jar
     |    |    |
     |    |    +---+plugin-cli-0.0.1.jar
     |    |
     |    +---+3.0.2
     |         +
     |         +---+ignite-core-3.0.2.jar
     |         |
     |         +---+plugin-cli-0.0.1.jar
     |
     +---+ignite-work
          +
          +---+ignite.modules={ignite-core,plugin-cli:0.0.1}
          |
          +---+ignite.classpath={ignite-core-3.0.2.jar:plugin-cli-0.0.1.jar}

Running Ignite

TBD

Cluster and local commands, management endpoint

Commands in the utility can be split into two groups: local commands (such as ignite run, ignite install), which can be executed without connection to a running cluster, and cluster commands (such as ignite baseline), which require connection to one of the ignite nodes. ignite cluster commands must have a straightforward and unambiguous mapping to the cluster management HTTP REST endpoints. With such a strict mapping we can achieve:

• easy implementation of all cluster commands at once which will solely require a list of available endpoints and arguments format
• command and argument autocompletion with possible autocompletion for dynamic arguments (for example, auto-complete node consistent ID for the ignite baseline command)
• the ability for a user to programmatically manage Ignite cluster from any platform/tool supporting HTTP (e.g. curl)

Pluggable commands

ignite utility should be extensible and allow plugging additional commands (both cluster and local). Upon the utility startup, it will scan the ${libsDir}/${version} folder for files with -cli suffix and search such files for command definitions. This allows a user to install additional commands using the same ignite install command. Update Apache Ignite configuration via configuration files to support external files (outside IGNITE_HOME) with external paths (outside IGNITE_HOME too) to persisted between version upgrades directories (work, log, etc.)

Risks and Assumptions

Proposed changes will require one time re-setup of Apache Ignite instance as well as constructing updated ignite.sh run command.
Next version upgrade will require only running the same ignite.sh run command from new version binary.

Discussion Links

Development list: IEP-52: Binary Delivery & Upgradability Enhancements

Reference Links

n/a

Tickets

n/a