Motivation

The IEP is intended to eliminate all the current limitations with cluster management and provide additional features which may be available out of the box.

Overview

All of Ignite Cluster management operations are presented as running compute tasks over the cluster. Depending on the management operation required to be executed some of the tasks are running on a single Ignite node only or performing full compute operation overall cluster nodes with reducing the computation result on the originating node.

For instance, single-node operations may look the following:

• running a built-in internal distributed operations e.g. IndexForceRebuiltTask;
• cancelling internal operations that are currently being run e.g.  VisorScanQueryCancelTask;

Distributed management compute tasks may look the following:

• collecting info about internal structures e.g. MetadataInfoTask;
• custom check operations over the cluster e.g. CheckIndexInlineSizesTask;

Limitations

Command API Exposure

Each time a new management task is added or is required to be added the developer should take into account the following things:

• a task must be exposed to the Ignite REST API (a new implementation of GridRestCommandHandler must be developed);
• a task must be exposed to the Ignite JMX API (a new MXBean interfaces and implementations must be added);
• a task must be exposed to the Ignite CLI (a new extension of AbstractCommand must be implemented);

Without handlers and abstractions mentioned above, a new management task won't be available for a user even if it persists in the node classpath (it's still possible to run it using the ignite thin client). Most of such handlers and abstractions look like a boilerplate code and must be covered by some internal command exposer. 

Command Plugin Extension

Ignite plugins have no control over the cluster management operations. The built-in management tasks can't be extended by Ignite running with custom plugins and even more, they can't be available through the standard Ignite's REST, JMX, CLI APIs.

Internal Binary Protocol

Ignite CLI uses under the hood its own internal communication protocol so-called Binary Rest Protocol which enables CLI tools to communicate with the existing Ignite cluster and run some compute tasks, perform cache operation, as well as has listeners of topology changes. An instance of the GridClient started each time a new management operation executes the same way as an instance of IgniteClient can do which in turn is a part of the official Ignite Binary Protocol. Taking into account that the Binary Rest Protocol which is used for the CLI tool fully undocumented looks like 

Security and Role Model

Ignite has some management tasks that don't obey the role model controlled by SecurityManager and has different execution flow depending on which execution context is set. For instance:

• a management task uses the internal API guarded by the SecurityPermission enum - both permissions for task execution by name and security permissions must be checked by SecurityManger (it seems only single permission must exist);
• a management task has GridTaskThreadContextKey#TC_SKIP_AUTH in its execution context - the authorization will be skipped at all;
• a management task has @GridInternal annotation - the task execution won't be covered by job execution events, so the external audit systems will skip the security issue;

All of the issues above require management tasks to be always being executed in the same manner with the same security context.

Design Issues

The following design issues bases on current management task implementation are present:

• commands and sub-commands have a common compute task return type, thus the result of running a particular sub-command will have unnecessary return data;
• some CLI implementations like the ignite-visor-console are running the Ignite node in a special daemon mode which doesn't look like an error-prone approach;
• it is not possible to add and run new management tasks at the cluster runtime (e.g. REPL mode for CLI tools can't be used);

Features

The following new features can be available out of the box with minimal additional source code changes:

• autogeneration AsciiDoc documentation for management commands with converting them to HTML and Unix man pages;
• integration with existing command-line frameworks like jcommander [2], picocli [3] etc.

Description

Design Model

The ComputeTask is a common way for building various cluster management tasks and execution via an Ignite thin client. 

Command

Management commands are always wrapped with the ProxyManagementTask. The management command may be executed on a single cluster node only or broadcasted to all nodes with reducing the execution results.

/** */
public interface IgniteCommand<T> extends IgniteCallable<T>, IgniteReducer<T, T> {
    /** {@inheritDoc} */
    @Override public default boolean collect(@Nullable T t) {
        return true;
    }

    /** {@inheritDoc} */
    @Override public default T reduce() {
        return null;
    }
}

/** */
@Command(name = "add",
    commandDescription = "Add nodes to baseline topology.")
public class BaselineAddCommand implements IgniteCallable<String> {
    /** Auto-injected Ignite instance. */
    @IgniteInstanceResource
    private IgniteEx ignite;

    /** Parameter will be injected on command instantiation. Default comma separation. */
    @Parameter(names = {"--nodes", "-n"}, description = "List of baseline nodes to add.")
    private List<String> consistentIds;

    /** {@inheritDoc} */
    @Override public String call() throws Exception {
        Collection<BaselineNode> baseline = ignite.cluster().currentBaselineTopology();
        Collection<ClusterNode> srvs = ignite.cluster().forServers().nodes();

        for (String consistentId : consistentIds) {
            ClusterNode node = F.find(srvs, null, new IgnitePredicate<ClusterNode>() {
                @Override public boolean apply(ClusterNode node) {
                    return node.consistentId().toString().equals(consistentId);
                }
            });

            if (node == null)
                throw new IllegalArgumentException("Node not found for consistent ID: " + consistentId);

            baseline.add(node);
        }

        ignite.cluster().setBaselineTopology(baseline);

        return consistentIds.toString();
    }
}

Roadmap

Phase-1

Phase-2

Risks and Assumptions

// Describe project risks, such as API or binary compatibility issues, major protocol changes, etc.

Discussion Links

// Links to discussions on the devlist, if applicable.

Reference Links

[1] https://www.mail-archive.com/dev@ignite.apache.org/msg49612.html

[2] https://jcommander.org/

[3] https://picocli.info/

Tickets