Motivation

Currently, Ignite 3 does not have a ready-made mechanism for delivering the required components to the user. The purpose of this IEP is to work out and create all kinds of options for packaging and distribution the required components of Ignite 3 to the user, taking into account the UX of various user platforms and OS.

Packaging

As ready-made resulting packages, it is planned to make only completely independent parts of Apache Ignite. At the moment only Ignite core part and Ignite CLI can work as standalone applications and these applications should be distributed as OS specific packages for all supported platforms. (Linux, Windows, MacOs). However, we will publish not only compiled packages, there is a need to publish individual parts of Apache Ignite that are not standalone applications, and we need to provide the user with the access to these parts. The prime candidate for this is the ignite-client with all its subparts (JDBC, SQL Api, etc Api). More about this will be described in paragraph Publishing.

Three packages will be provided:

1. ignite-cli — Ignite 3 CLI tool. Used if one wants to manage a remote cluster.
2. ignite-core — Ignite 3 core artifacts. Used if one wants to start nodes locally.
3. ignite —  meta package that includes both ignite-cli and ignite-core.
4. ignite-odbc — package with ignite-odbc libs. 
5. ignite-native-libs — 
6. ignite-jdbc  — jdbc package 

Every package will be available as a downloadable ZIP file, and as OS-specific packages (RPM, DEB, Brew …).

ignite-cli

This package can only be used to connect to a remote (or local) cluster. Once the package is installed, the user gets access to the Ignite CLI tool via the ignite command.

The package includes:

1. CLI tool executables for Linux and Windows
2. LICENSE and NOTICE files per Apache requirements

NOTE: It is possible to build an ignite-cli executable with GraalVM native-image technology. This approach can provide a significant performance boost during CLI startup. This performance increase can be tested and presented to the community as part of this work.

ignite-core

This package provides an ability to run nodes on the local server. It includes:

1. All required Ignite core artifacts with any dependencies
2. Ignite-core or ignite-server (start, stop, restart)  script, both for Linux and Windows.
3. LICENSE and NOTICE files per Apache requirements

Scripts can be used by a user directly, or by OS-level services. Mix-and-match of these two approaches should be disallowed.

ignite

This is a meta package that includes everything listed for the other two packages.

ignite-odbc

TBD. Currently IEP for odbc not ready.

ignite-native-libs

TBD.

Open API spec

TBD

Publishing

The issue of publishing assembled packages should be highlighted in a separate paragraph.

Each package should be published and available to install for all supported platforms by native OS way. It means that the installation process should respect all UX of each OS :

1. We should provide the possibility to install each component with the default package manager (apt-get, yum, brew, etc.) of the OS.
2. Install, upgrade, update, uninstall process should require all rules of OS guidelines. For example remove all temp files after uninstall and not change these files in the upgrade process.

It is also necessary to support the publication of compiled images as zip files for the possibility of installation on those platforms where there are no package managers. For each supported architecture, it is necessary to build and publish a zip file with everything necessary for the application to fully work (except for the third party dependencies, which should be installed as additional packages, but this list should be attached to the REQUIRED.txt file).

Code Examples

Code examples should be moved to a separate Git repository and also provided as a separate ZIP download for every release.

Examples are NOT included in any of the packages described above.

TDB

Gradle

In the context of packaging work it would be useful to rewrite Apache Ignite to a Gradle build system. This work should be done before all packaging staff, we also need to verify that the build process is correctly working as previously. Also need to adapt all CI scripts for the new build system and scripts.

These several point about what can be improved by Gradle using:

1. Various Gradle plugins that can make the build process easier.  Rpm/Deb, brew, SDKMan, Docker, etc.
2. Currently, the Apache Ignite project contains many different validation scripts that need to be run manually or in TeamCity's automation scripts. Instead of all this, it can be unified in the gradle build scripts in order to be able to locally fully reproduce the build with CI.
3. Since we have 3 different packaging targets, all packaging logic can be unified in 1 gradle script and just used in target modules.

Target package formats

Each installation method will be described here. As described above, each method must fully respect all the features of the user experience of the platform. Because of this, care should be taken in attempting to unify user scripts between different installation methods.

NOTE: All formats and running options MUST provide possibility to run multiple instances of ignite-core module.

Zip archive

The file includes:

1. All executables for Linux and Windows and MacOS
2. LICENSE and NOTICE files per Apache requirements

Installation:

1. Download the zip file
2. Unzip into a folder
3. Add executable into a PATH (optional, if yes – ask user)

Uninstallation v1:

1. Stop ignite processes
2. Remove a folder

Uninstallation v2:

1. Cd into the folder
2. Run uninstall script

Upgrade v1:

1. Download the zip file
2. Unzip into a folder
3. Add executable into a PATH (optional) 

Upgrade v2:

1. Download the zip file
2. Cd into the folder with previous version
3. Run upgrade executable and provide the zip file as an argument

Upgrade process should not remove data files.

Zip archive can be built with a maven/gradle plugin or a script.

RPM/DEB packages

The package includes:

1. All executables for Linux
2. LICENSE and NOTICE files per Apache requirements

Installation:

1. Download RPM/DEB package
2. Run rpm -i <package> / apt install <path-to-deb>/ dpkg -i <package>
3. After installation the executable is represented as system service (service file) and can be run with system ignite start command. 

Uninstallation:

1. Run rpm -e <package> / dpkg --r <package>

Upgrade:

1. Run rpm -U <package> / dpkg -i <package>

NOTE:: upgrade process should not remove data files.

RPM/DEB archive can be built by plugin or https://github.com/jreleaser/jreleaser

Yum/apt-get package managers

Apache Ignite should have a RPM/Deb repository with all builded packages. 

This is a common approach in the Linux community and is the standard way to install an application on a system.

1. User add custom repository 
1. yum install https://yum-rep-address/repo-info.rpm
2. add-apt-repository https://repo-address
2. Install required package
1. yum install ignite ignite-cli
2. apt install ignite ignite-cli

NOTE: Also in the future it is necessary to consider the possibility of adding our packages to the standard repositories of common distributions.

Homebrew package

Homebrew uses Formula to install the software. Formula is a Ruby class that describes the installation process. Formula should be placed in GitHub and have a link to the zip archive.

There is also a way to create Formula via a gradle plugin https://github.com/jreleaser/jreleaser

Installation:

1. Add custom github repo to the brew: brew tap apache/ignite-3
2. Run brew install <package>
3. After installation the executable is represented as command line app (ignite (CLI), ignite-core or ignite-server (start, stop, restart))

Uninstallation:

1. Run brew uninstall <package>

Upgrade:

1. Run brew upgrade <package>

SDKMan

SDKman is a multiplatform package manager with supported platforms: 

• LINUX_64
• LINUX_ARM64
• LINUX_32
• LINUX_ARM32SF
• LINUX_ARM32HF
• MAC_OSX
• MAC_ARM64
• WINDOWS_64

A lot of product publishing in SDKMan java, kotlin, scala, maven, gradle, sbt, Scala CLI, Quarkus CLI, Apache ActiveMQ, Flink, Spark, etc.

Installation:

sdk install ignite 3.0.0-SNAPSHOT (or another version)

Uninstallation:

sdk uninstall ignite ${version} 

List:

     sdk list ignite (or ignite-cli)

Publishing process is described here and it's not much different from other package managers.

Also publishing process can be simplified by using https://github.com/jreleaser/jreleaser or https://github.com/sdkman/sdkman-vendor-gradle-plugin

Docker

For the ignite-core module we also should build a Docker image which can be used for easy Apache Ignite startup.

This image should contain ready for use Apache Ignite with all required dependencies. After the image is created Apache Ignite should be started and ready for use. By default all ports of Ignite server should be forwarded as is and no additional changes required.

Also https://jreleaser.org/guide/latest/configuration/packagers/docker.html can simplify the process of docker image building.

Summary

As you can see https://github.com/jreleaser/jreleaser can help us with publishing setup infrastructure and cover mostly all cases except WinGet. This is a good reason to try to use it as the only required dependency to unify how different packages are built.

Also https://jreleaser.org/guide/latest/examples/micronaut-cli-app.html provides native support of Micronaut CLI project which is Apache Ignite CLI.

Risks and Assumptions

// Describe project risks, such as API or binary compatibility issues, major protocol changes, etc.

Discussion Links

// Links to discussions on the devlist, if applicable.

Reference Links

// Links to various reference documents, if applicable.

Tickets

// Links or report with relevant JIRA tickets.