Motivation

Currently, many Ignite components use the following approach: internal data (context) is attached to an operation and is accessed during its execution.
This approach is often implemented using ThreadLocal variables.

...

Examples of context propagation problems:

• IGNITE-23958
• IGNITE-16405
• IGNITE-15101

This IEP is intended to provide a robust mechanism for propagating arbitrary operation context that preserves the convenience of the ThreadLocal approach while minimizing the risk of operation context loss in a multithreaded environment.   

Description

The following steps are proposed to resolve the described problem:

1.  Create an ThreadLocal storage (Context) with the following properties
           a) Ignite node-independent
           b) stores arbitrary attributes and their corresponding values
           c) provides the ability to update attribute values
           d) provides the ability to automatically undo the last Context update, restoring previous attribute values, after the corresponding operation completes
           e) provides the ability to creates a snapshot of stored attributes and their values
           f) provides the ability to restores attribute values from snapshot to Context in another thread
   
   This will help create a unified mechanism for attaching arbitrary data to an operation for its duration, as well as the ability to move context attribute values ​​between threads.
   
   
2. Integrate the ability to capture Context Attributes values via snapshot creation to Ignite into Thread Pools Pool and Ignite Futures classes.
   Methods like IgniteThreadPooExecutor#submit and IgniteFuture#listen must It will require to create a special wrappers over existing classes.
   Methods such as ExecutorSecvice#submit or IgniteFuture#listen should automatically capture and save the state of the Context state along with the closure that may could potentially be executed in on another thread, and restore the saved Context before executing it.
   

Risks and Assumptions

1. 
   The following Ignite and Java classes should be considered:
   
   1. IgniteThreadPoolExecutor
   2. StripedExecutor
   3. StripedThreadPoolExecutor
   4. ForkJoinPool
   5. ScheduledThreadPoolExecuror
   6. ForkJoinPool.commonPool
   7. GridFutureAdapter
   8. CompletableFuture
      
      
2. Add special rules for the static code analyzer to prevent the use of  classes related to asynchronous execution that do not support automatic Context capture/restore. It seems that even the creation of new Threads in Ignite internal code should be limited by default - but this is a matter for discussion..
   We should also support an ability to exclude classes and and modules from this check - e.g. the mentioned check can be skipped for thin client related code.
   
   
3. Research and implement a mechanism for propagating context between remote Ignite nodes to cover the case where an operation continues to execute on a remote node. The question of whether the entire Сontext should be transferred or only a specific attribute remains open, as transferring the entire Сontext between nodes may result in increased network overhead.

Example of ThreadLocal storage API:

public class Context {   
    /**
     * Retrieves value associated with specified attribute by accessing Context bound to the thread this method is
     * called from. If no value is explicitly associated with specified attribute, {@link ContextAttribute#initialValue()}
     * is returned.
     *
     * @param attr Context Attribute.
     * @return Context Attribute Value.
     */
    @Nullable public static <T> T get(ContextAttribute<T> attr);

    /**
     * Updates the value of the specified attribute for the Context bound to the thread this method is called from.
     *
     * @param attr Context Attribute.
     * @return Scope instance that, when closed, undoes the applied update. It is crucial to undo all applied Context
     * updates to free up thread-bound resources and avoid memory leaks, so it is highly encouraged to use a
     * try-with-resource block to close the returned Scope. Note, updates must be undone in the same order and in the
     * same thread they were applied.
     */
    public static <T> Scope set(ContextAttribute<T> attr, T val);   

    /**
     * Creates Snapshot of all attributes and their corresponding values stored in the Context bound to the thread this
     * method is called from.
     *
     * @return Context Snapshot.
     */
    public static ContextSnapshot createSnapshot();

    /**
     * Restores values of all attributes for Context bound to the thread this method is called from.
     *
     * @param snp Context Snapshot.
     * @return Scope instance that, when closed, undoes the applied operation. It is crucial to undo all applied Context
     * updates to free up thread-bound resources and avoid memory leaks, so it is highly encouraged to use a
     * try-with-resource block to close the returned Scope. Note, updates must be undone in the same order and in the
     * same thread they were applied.
     */
    public static Scope restoreSnapshot(ContextSnapshot snp);
}

   /** */
   private static final ContextAttribute<String> ATTR = ContextAttribute.newInstance("initial-value");
 
   /** */
   public static void main(String[] args) {
       ContextSnapshot snapshot;
 
       try (Scope scope = Context.set(ATTR, "test")) {
           String attrVal = Context.get(ATTR);
       }
 
       ContextSnapshot snapshot = Context.createSnapshot();
 
       try (Scope scope = Context.restoreSnapshot(snapshot)) {
           String attrVal = Context.get(ATTR);
       }
   }

Risks and Assumptions

The proposed improvement allows to largely eliminate the problem of losing the context of an operation in a multi-threaded environment, but, unfortunately, does not solve this problem completely.
External libraries may provide the ability to submit code for multithreaded execution, but obviously do not support context propagation. Like JDK mechanism - java.util.stream.BaseStream#parallel. Misuse of such mechanisms can still result in loss of operation context and should be treated with caution.

Ubiquitous support for context capture and recovery may impact performance and GC. Although preliminary tests did not reveal any noticeable impact// Describe project risks, such as API or binary compatibility issues, major protocol changes, etc.

Discussion Links

// Links to discussions on the devlist, if applicable.

Reference Links

https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/ScopedValue.html

// Links to various reference documents, if applicable.

Tickets

https://github.com/grpc/grpc-java/blob/master/api/src/context/java/io/grpc/Context.java

https://github.com/open-telemetry/opentelemetry-java/tree/main/context/src/main/java/io/opentelemetry/context

https://projectreactor.io/docs/core/release/reference/advanched-contextPropagation.html

https://github.com/micrometer-metrics/context-propagation

https://logback.qos.ch/manual/mdc.html

Tickets