...

Motivation

The IEP is intended to eliminate all the current limitations with cluster management and provide additional features which may be available out of the boxsimplify, standardise, unifies development of Ignite management commands by providing declarative API for arguments and unified command invokers via various protocols - CLI, JMX, REST.

Overview

All of Ignite Cluster management operations are presented as running compute tasks over the cluster. Depending on the management operation required to be executed some of the tasks are running on a single Ignite node only or performing full compute operation overall cluster nodes with reducing the computation result on the originating node.

...

Without handlers and abstractions mentioned above, a new management task won't be available for a user even if it persists in the node classpath (it's still possible to run it using the ignite thin client). Most of such handlers and abstractions look like a boilerplate code and must be covered by some internal command exposer.

There are no any sufficient abstractions in any protocol and each protocol has (or has no if you lucky) slightly different logic.

Moreover, format of arguments, help messages and other things are not standardized and have different logic from command to command. 

Command Plugin Extension

Ignite plugins have no control over the cluster management operations. The built-in management tasks can't be extended by Ignite running with custom plugins and even more, they can't be available through the standard Ignite's REST, JMX, CLI APIs.

Internal Binary Protocol

Ignite CLI uses under the hood its own internal communication protocol so-called Binary Rest Protocol which enables CLI tools to communicate with the existing Ignite cluster and run some compute tasks, perform cache operation, as well as has listeners of topology changes. An instance of the GridClient started each time a new management operation executes the same way as an instance of IgniteClient can do which in turn is a part of the official Ignite Binary Protocol. Taking into account that the Binary Rest Protocol which is used for the CLI tool fully undocumented looks like 

Security and Role Model

Ignite has some management tasks that don't obey the role model controlled by SecurityManager and has different execution flow depending on which execution context is set. For instance:

• a management task uses the internal API guarded by the SecurityPermission enum - both permissions for task execution by name and security permissions must be checked by SecurityManger (it seems only single permission must exist);
• a management task has GridTaskThreadContextKey#TC_SKIP_AUTH in its execution context - the authorization will be skipped at all;
• a management task has @GridInternal annotation - the task execution won't be covered by job execution events, so the external audit systems will skip the security issue;

All of the issues above require management tasks to be always being executed in the same manner with the same security context.

Design Issues

The following design issues based on current management task implementation are present:

• commands and sub-commands have a common compute task return type, thus the result of running a particular sub-command will have unnecessary return data;
• some CLI implementations like the ignite-visor-console are running the Ignite node in a special daemon mode which doesn't look like an error-prone approach;
• it is not possible to add and run new management tasks at the cluster runtime (e.g. REPL mode for CLI tools can't be used);

Features

The following new features can be available out of the box with minimal additional source code changes:

• autogeneration AsciiDoc documentation for management commands with converting them to HTML and Unix man pages;
• integration with existing command-line frameworks like jcommander [2], picocli [3] etc.

Description

Design Model

Command Execution

The ComputeTask is a common way for building various cluster management commands with the ability to execute them via Ignite Binary Protocol. Taking into account the current limitations mentioned above the following must be a part of design solution to create a common internal management API:

1. A ProxyManagementTask - an entry point for each management request through the thin client API.
2. Input arguments to find the required command by given path.
3. Input argements to execute the corresponding command with. This is a map of parameters with String  as a key, and String  or String[]  as a value.
4. The output execution result – BinaryObject. It may be formatted to different string results depending on what type of client is used (e.g. REST, CLI, JMX). 

Command Registration

1. Static Registration
2. ServiceLoader
3. Annotation Scanner

Command Execution Model

...

Description

This IEP is primary focused on providing good abstraction for command invocation which includes:

• Internal framework to declarative command definition.
• Migration of existing commands to new framework without any changes in public behavior(if possible).
• Creation of commands registry which provide unified access to all commands known by Ignite. 
• Creation of invokers for all popular protocols. Each invoker must be able to invoke any of the command based on definition:
  • CLI - control.sh.
  • JMX.
  • REST - Open API bindings and REST endpoint.
• Automatic documentation creation:
  • Ascii doc during release preparation.
  • man/tldr pages.

Keeping in mind that command will be described in declarative way all newly created commands will be automatically available and used by all of the features above. 

Command Execution

Flow of command execution:

1. Determine specific command.
2. Parsing of arguments. Design assumes that every agrument will be presented as string.
3. Filtering nodes based on arguments.
4. Command execution.
5. Print results. 

Design

The following design principles are proposed:

1. Declarative command description:
   1. "path" to execute command derived from class name.
      1. SystemViewCommand resolved to path ./control.sh --system-view ... 
      2. BaselineAddCommand resolved to path ./control.sh --baseline-add ...
   2. Command argument class and fields described with annotations. This allow to fill command argument using reflection.
2. All arguments received as strings.
3. Results is human readable string.
4. Command are stateless.
5. Each command can have subcommands.
6. All commands united in command registry.
7. Command registry is only way to obtain command instance. 

API

...

Command Interface

Management commands are always wrapped with the ProxyManagementTask. The management command may be executed on a single cluster node only or broadcast to all nodes with reducing the execution results.

//** */
public interface IgniteCommand<T> extends IgniteCallable<T>, IgniteReducer<T, T> {
    /** {@inheritDoc} */
    @Override public default boolean collect(@Nullable T t) A - argument type.
// R - result type.
// T - task type. Task implements actual command logic.

public interface Command<A extends IgniteDataTransferObject, R, T extends ComputeTask<VisorTaskArgument<A>, R>> {
    public String description();

    return truepublic Class<A> args();

    public Class<T> }task();

    public  /** {@inheritDoc} */
    @Override public default T reduce() {void printResult(IgniteDataTransferObject arg, Object res, Consumer<String> printer);

    boolean experimental();

    boolean confirmable();

   return null;
    }Collection<UUID> nodes(Collection<UUID> nodes, A arg);
}

/** */
@Command(name@OneOf(value = {"addnodeIds",
 "nodeId",   commandDescription"allNodes"}, optional = "Add nodes to baseline topology."true)
public class BaselineAddCommandSystemViewCommandArg implementsextends IgniteCallable<String>IgniteDataTransferObject {
    /** Auto-injected Ignite instance. */@Positional
    @IgniteInstanceResource
@Argument(description = "Name of privatethe IgniteEx ignite;

    /** Parameter willsystem view which content should be injected on command instantiation. Default comma separation. */
    @Parameter(names = {"--nodes", "-n"}, description = "List of baseline nodes to add.")printed." +
        " Both \"SQL\" and \"Java\" styles of system view name are supported" +
    private List<String> consistentIds;

  "  /** {@inheritDoc} */
    @Override public String call() throws Exception {(e.g. SQL_TABLES and sql.tables will be handled similarly)")
    private String systemViewName;

  Collection<BaselineNode> baseline = ignite.cluster().currentBaselineTopology();@Argument(
        Collection<ClusterNode>description srvs= = ignite.cluster().forServers().nodes();

        for (String consistentId : consistentIds) {
            ClusterNode node = F.find(srvs, null, new IgnitePredicate<ClusterNode>() {"ID of the node to get the system view from (deprecated. Use --node-ids instead). If not set, random node will be chosen",
        optional = true
    )
    private UUID nodeId;

      @Override public boolean apply(ClusterNode node) {@Argument(
        description = "Comma-separated list of nodes IDs to get the system  return node.consistentId().toString().equals(consistentId);
        view from. If not set, random node will be chosen",
        }
optional = true
    )
    private UUID[] })nodeIds;

            if (node == null)@Argument(
        description = "Get the system view from all thrownodes. new IllegalArgumentException("Node not found for consistent ID: " + consistentId);

If not set, random node will be chosen",
        optional    baseline.add(node);= true
        }
)
    private    ignite.cluster().setBaselineTopology(baseline)boolean allNodes;


    // The rest of returnthe consistentIds.toString();
    }code.
}

Roadmap

Phase-1

The following changes must be introduced to allow new command creation and migration processes:

• annotations to mark up command classes and annotation-based scanner to fill-up the command registry;internal framework to create commands.
• command registry (available for clients and for server nodes);
• input string parsers for CLI, REST, JMX interfaces to allow command migration to a new CommandRegistry;proxy compute task for commands management through the GridTaskExecutor;
• migration of all commands to new framework
• deprecation of existing JMX beans and REST API.

Phase-2

To be done.

Risks and Assumptions

...

• ability to register commands provided by plugins. 
• automatic documentation generation
  • html
  • man pages

Phase-3

• thin client cli implementation.
• deprecation of control.sh

Risks and Assumptions

Discussion Links

// Links to discussions on the devlist, if applicable.

...