Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

IgniteClassPath properties:

  • Name - unique string identifier (app_v1, 01_07_2025, etc).
  • Status - lifecycle status: CREATING, READY, REMOVING, etc.
  • Set of libs and resources associated with IgniteClassPath.
  • Version - version of ICP in the form of x.y.z (1.0.3, for example). 
    max version can be referenced as "latest".
  • Each instance available on each Ignite node, maybe on demand.
  • Can be created and removed runtime - without any restarts.
  • Instances independent from each other.

...

$ ./bin/flink run -C /path/to/dependency.jar -c com.example.MyJob my-flink-job.jar

Design

There are two perspectives for ICP operations: a user perspective and a cluster perspective.

From a user perspective 4 main operations are allowed for ICP:

  • create.
  • safe remove from the cluster. Will wait until all code running on the top of ICP stops.
  • unsafe remove. Will forcefully remove ICP regardless of code currently running. 
  • get ICP status in the cluster: list of all deployment units in the cluster with their statuses (will be defined below).
  • get ICP status on a specific node. The main scenario for this operation is troubleshooting.

From a cluster perspective a deployment unit could be deployed on the cluster, but it could be not deployed on a particular node. So additional operation is required: create ICP on the target node (on demand).

Directories

All ICP must be placed in the ICP base directory which is a subdirectory under Ignite work directory:

<ignite_work_dir>/icp

For each ICP a directory should be created under the base directory. The name of this directory must be the same as ICP name and a nested directory for a particular version must be created. 

Example:

- icp
    - foo.example.job 
        - 1.0.0
        - 1.0.1
    - foo.example.task
        - 1.0.0
        - 2.0.0

Risks and Assumptions

We assume that p2p and deployment SPI will be removed from Ignite.

...