...

Motivation

Currently, Ignite 3 does not have a ready-made mechanism for delivering the required components to the user. The purpose of this IEP is to work out and create all kinds of options for packaging and distributing the required components of Ignite 3 to the user, taking into account the UX of various user platforms and OS.

Packaging

As ready-made resulting packages, it is planned to make only completely independent parts of Apache Ignite. At the moment only Ignite core part and Ignite CLI can work as standalone applications and these applications should be distributed as OS-specific packages for all supported platforms. (Linux, Windows, MacOs). However, we will publish not only compiled packages, but there is also a need to publish individual parts of Apache Ignite that are not standalone applications, and we need to provide the user with access to these parts. The prime candidate for this is the ignite-client with all its subparts (JDBC, SQL Api, etc Api). More about this will be described in paragraph Publishing.

...

Every package will be available as a downloadable ZIP file, and as OS-specific packages (RPM, DEB, Brew …).

ignite3-cli

This package can only be used to connect to a remote (or local) cluster. Once the package is installed, the user gets access to the Ignite CLI tool via the ignite command.

...

NOTE: It is possible to build an ignite3-cli executable with GraalVM native-image technology. This approach can provide a significant performance boost during CLI startup. This performance increase can be tested and presented to the community as part of this work.

ignite3-db

This package provides the ability to run nodes on the local server. It includes:

...

Scripts can be used by a user directly, or by OS-level services. Mix-and-match of these two approaches should be disallowed.

ignite3

This is a meta package that includes everything listed for the other two packages.

ignite-jdbc

Separate package of shaded ignite JDBC with all dependencies. Speaking of fat jar: jdbc depends on netty and msgpack, which is a quite popular libraries. To avoid conflict of versions in downstream projects it would be better to shade dependecies. This package should be also published to maven central.

Publishing

The issue of publishing assembled packages should be highlighted in a separate paragraph.

...

It is also necessary to support the publication of compiled images as zip files for the possibility of installation on those platforms where there are no package managers. For each supported architecture, it is necessary to build and publish a zip file with everything necessary for the application to fully work (except for the third-party dependencies, which should be installed as additional packages, but this list should be attached to the REQUIRED.txt file).

Gradle

In the context of packaging work, it would be useful to rewrite Apache Ignite build system to a Gradle build system. This work should be done before all packaging staff, we also need to verify that the build process is correctly works as previously. Also need to adapt all CI scripts for the new build system and scripts.

...

1. Various Gradle plugins can make the build process easier. Rpm/Deb, brew, SDKMan, Docker, etc.
2. Currently, the Apache Ignite project contains many different validation scripts that need to be run manually or in TeamCity's automation scripts. Instead of all this, it can be unified in the Gradle build scripts to be able to locally fully reproduce the build with CI.
3. Since we have 3 different packaging targets (ignite3-db, ignite3-cli, ignite3) all packaging logic can be unified in 1 Gradle script and just used in target modules.

Target package formats

Each installation method will be described here. As described above, each method must fully respect all the features of the user experience of the platform. Because of this, care should be taken in attempting to unify user scripts between different installation methods.

NOTE: All formats and running options MUST provide the possibility to run multiple instances of the ignite3-db module.

Zip archive

The file includes:

1. All ignite artifacts and executables for Linux and Windows and MacOS
2. LICENSE and NOTICE files per Apache requirements

...

Zip archive can be built with a Gradle plugin.

RPM/DEB packages

The package includes:

1. All executables for Linux
2. LICENSE and NOTICE files per Apache requirements

...

RPM/DEB archive can be built by plugin or https://github.com/jreleaser/jreleaser

Yum/apt-get package managers

Apache Ignite should have an RPM/Deb repository with all built packages. 

...

NOTE: Also in the future, it is necessary to consider the possibility of adding our packages to the standard repositories of common distributions.

Homebrew package

Homebrew uses Formula to install the software. Formula is a Ruby class that describes the installation process. Formula should be placed on GitHub and have a link to the zip archive.

...

Upgrade:

1. Run brew upgrade ignite3 

SDKMan

SDKman is a multiplatform package manager with supported platforms: 

...

Also publishing process can be simplified by using https://github.com/jreleaser/jreleaser or https://github.com/sdkman/sdkman-vendor-gradle-plugin

Docker

For the ignite3-db module, we also should build a Docker image that can be used for easy Apache Ignite startup.

...

Also, https://jreleaser.org/guide/latest/configuration/packagers/docker.html can simplify the process of docker image building.

Summary

As you can see https://github.com/jreleaser/jreleaser can help us with publishing setup infrastructure and cover all cases. This is a good reason to try to use it as the only required dependency to unify how different packages are built.

Also, https://jreleaser.org/guide/latest/examples/micronaut-cli-app.html provides native support for the Micronaut CLI project which is Apache Ignite CLI.

Risks and Assumptions

Mostly, there are no risks.The main critical part of the changes is the transition of the build process to Gradle.This moment should be separately tested, and the transfer of CI to a new build system should be made after a complete ready-check of the build gradle.

Discussion Links

TDB

Tickets