Motivation

...

Examples of context propagation problems:

• IGNITE-23958
• IGNITE-16405
• IGNITE-15101

This IEP is intended to provide a robust mechanism for propagating arbitrary operation context that preserves the convenience of the ThreadLocal approach while minimizing the risk of operation context loss in a multithreaded environment.   

...

1.  Create an ThreadLocal storage (Context) with the following properties
           a) Ignite-independent
           b) stores arbitrary attributes and their corresponding values
           c) provides the ability to update attribute values
           d) provides the ability to automatically undo the last Context update, restoring previous attribute values, after the corresponding operation completes
           e) provides the ability to creates a snapshot of stored attributes and their values
           f) provides the ability to restores attribute values from snapshot to Context in another thread
   
   This will help create a unified mechanism for attaching arbitrary data to an operation for its duration, as well as the ability to move context attribute values ​​between threads.
   
   
2. Integrate the ability to capture Context Attributes values via snapshot creation to Ignite into Thread Pools Pool and Ignite Futures classes.
   Methods like IgniteThreadPooExecutor#submit and IgniteFuture#listen must It will require to create a special wrappers over existing classes.
   Methods such as ExecutorSecvice#submit or IgniteFuture#listen should automatically capture and save the state of the Context state along with the closure that may could potentially be executed in on another thread, and restore the saved Context before executing it.
   
   The following Ignite and Java classes should be considered:
   
   1. IgniteThreadPoolExecutor
   2. StripedExecutor
   3. StripedThreadPoolExecutor
   4. ForkJoinPool
   5. ScheduledThreadPoolExecuror
   6. ForkJoinPool.commonPool
   7. GridFutureAdapter
   8. CompletableFuture
      
      
3. Add special rules for the static code analyzer to prevent the use of multithreading-related classes that do not support automatic Context capture/restore.

Example of ThreadLocal storage API:

public class Context {   
    /**
     * Retrieves value associated with specified attribute by accessing Context bound to the thread this method is
     * called from. If no value is explicitly associated with specified attribute, {@link ContextAttribute#initialValue()}
     * is returned.
     *
     * @param attr Context Attribute.
     * @return Context Attribute Value.
     */
    @Nullable public static <T> T get(ContextAttribute<T> attr);

    /**
     * Updates the value of the specified attribute for the Context bound to the thread this method is called from.
     *
     * @param attr Context Attribute.
     * @return Scope instance that, when closed, undoes the applied update. It is crucial to undo all applied Context
     * updates to free up thread-bound resources and avoid memory leaks, so it is highly encouraged to use a
     * try-with-resource block to close the returned Scope. Note, updates must be undone in the same order and in the
     * same thread they were applied.
     */
    public static <T> Scope set(ContextAttribute<T> attr, T val);   

    /**
     * Creates Snapshot of all attributes and their corresponding values stored in the Context bound to the thread this
     * method is called from.
     *
     * @return Context Snapshot.
     */
    public static ContextSnapshot createSnapshot();

    /**
     * Restores values of all attributes for Context bound to the thread this method is called from.
     *
     * @param snp Context Snapshot.
     * @return Scope instance that, when closed, undoes the applied operation. It is crucial to undo all applied Context
     * updates to free up thread-bound resources and avoid memory leaks, so it is highly encouraged to use a
     * try-with-resource block to close the returned Scope. Note, updates must be undone in the same order and in the
     * same thread they were applied.
     */
    public static Scope restoreSnapshot(ContextSnapshot snp);
}

   /** */
   private static final ContextAttribute<String> ATTR = ContextAttribute.newInstance("initial-value");
 
   /** */
   public static void main(String[] args) {
       ContextSnapshot snapshot;
 
       try (Scope scope = Context.set(ATTR, "test")) {
           String attrVal = Context.get(ATTR);
       }
 
       ContextSnapshot snapshot = Context.createSnapshot();
 
       try (Scope scope = Context.restoreSnapshot(snapshot)) {
           String attrVal = Context.get(ATTR);
       }
   }

Risks and Assumptions

// Describe project risks, such as API or binary compatibility issues, major protocol changes, etcThe proposed improvement allows to largely eliminate the problem of losing the context of an operation in a multi-threaded environment, but, unfortunately, does not solve this problem completely.
External libraries may provide the ability to submit code for multithreaded execution, but obviously do not support context propagation. Like JDK mechanism - java.util.stream.BaseStream#parallel. Misuse of such mechanisms can still result in loss of operation context and should be treated with caution.

Ubiquitous support for context capture and recovery may impact performance and GC. Although preliminary tests did not reveal any noticeable impact.

Discussion Links

// Links to discussions on the devlist, if applicable.

Reference Links

https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/ScopedValue.html

https://github.com/open-telemetry/opentelemetry-java/

...

tree/main/context/src/main/java/io/opentelemetry/context

https://projectreactor.io/docs/core/release/reference/advanched-contextPropagation.html

https://github.com/micrometer-metrics/context-propagation

https://logback.qos.ch/manual/mdc.html

Tickets

Tickets

// Links or report with relevant JIRA tickets.