IDIEP-143
Author
Sponsor
Created

  

Status
DRAFT

Motivation

Currently, many Ignite components use the following approach: internal data (context) is attached to an operation and is accessed during its execution.
This approach is often implemented using ThreadLocal variables.

Examples of components that use the approach described above, along with their corresponding context classes:

and so on.

While the ThreadLocal approach eliminates the need for developers to manually propagate the context, it can also lead to errors when an operation is executed in a multithreaded environment.
Consider a situation in which an operation begins on one thread and continues on another. This can occur, for example, if a subtask is explicitly submitted to a thread pool or if IgniteFuture#listen is used to suspend the operation and resume it on the thread that completes the IgniteFuture.
In all such cases, the developer must manually restore the context in the thread that continues executing the operation; otherwise, the operation context stored in the ThreadLocal variable will be lost.

In some cases, losing the context associated with an operation can be critical. For example, losing the SecurityContext, which stores information about the initiator, can lead to incorrect authorization decisions and inaccurate auditing.

Examples of context propagation problems:

This IEP is intended to provide a robust mechanism for propagating arbitrary operation context that preserves the convenience of the ThreadLocal approach while minimizing the risk of operation context loss in a multithreaded environment.   

Description

The following steps are proposed to resolve the described problem:

  1.  Create an ThreadLocal storage (Context) with the following properties
            a) Ignite node-independent
            b) stores arbitrary attributes and their corresponding values
            c) provides the ability to update attribute values
            d) provides the ability to automatically undo the last Context update, restoring previous attribute values, after the corresponding operation completes
            e) provides the ability to creates a snapshot of stored attributes and their values
            f) provides the ability to restores attribute values from snapshot to Context in another thread

    This will help create a unified mechanism for attaching arbitrary data to an operation for its duration, as well as the ability to move context attribute values ​​between threads.

  2. Integrate the ability to capture Context Attributes values via snapshot creation into Thread Pool and Futures classes. It will require to create a special wrappers over existing classes.
    Methods such as ExecutorSecvice#submit or IgniteFuture#listen should automatically capture and save the state of the Context along with the closure that could potentially be executed on another thread, and restore the saved Context before executing it.

    The following Ignite and Java classes should be considered:
    1. IgniteThreadPoolExecutor
    2. StripedExecutor
    3. StripedThreadPoolExecutor
    4. ForkJoinPool
    5. ScheduledThreadPoolExecuror
    6. ForkJoinPool.commonPool
    7. GridFutureAdapter
    8. CompletableFuture

  3. Add special rules for the static code analyzer to prevent the use of  classes related to asynchronous execution that do not support automatic Context capture/restore. It seems that even the creation of new Threads in Ignite internal code should be limited by default - but this is a matter for discussion..
    We should also support an ability to exclude classes and and modules from this check - e.g. the mentioned check can be skipped for thin client related code.

  4. Research and implement a mechanism for propagating context between remote Ignite nodes to cover the case where an operation continues to execute on a remote node. The question of whether the entire Сontext should be transferred or only a specific attribute remains open, as transferring the entire Сontext between nodes may result in increased network overhead.


Example of ThreadLocal storage API:

Example of ThreadLocal storage API:
public class Context {   
    /**
     * Retrieves value associated with specified attribute by accessing Context bound to the thread this method is
     * called from. If no value is explicitly associated with specified attribute, {@link ContextAttribute#initialValue()}
     * is returned.
     *
     * @param attr Context Attribute.
     * @return Context Attribute Value.
     */
    @Nullable public static <T> T get(ContextAttribute<T> attr);

    /**
     * Updates the value of the specified attribute for the Context bound to the thread this method is called from.
     *
     * @param attr Context Attribute.
     * @return Scope instance that, when closed, undoes the applied update. It is crucial to undo all applied Context
     * updates to free up thread-bound resources and avoid memory leaks, so it is highly encouraged to use a
     * try-with-resource block to close the returned Scope. Note, updates must be undone in the same order and in the
     * same thread they were applied.
     */
    public static <T> Scope set(ContextAttribute<T> attr, T val);   

    /**
     * Creates Snapshot of all attributes and their corresponding values stored in the Context bound to the thread this
     * method is called from.
     *
     * @return Context Snapshot.
     */
    public static ContextSnapshot createSnapshot();

    /**
     * Restores values of all attributes for Context bound to the thread this method is called from.
     *
     * @param snp Context Snapshot.
     * @return Scope instance that, when closed, undoes the applied operation. It is crucial to undo all applied Context
     * updates to free up thread-bound resources and avoid memory leaks, so it is highly encouraged to use a
     * try-with-resource block to close the returned Scope. Note, updates must be undone in the same order and in the
     * same thread they were applied.
     */
    public static Scope restoreSnapshot(ContextSnapshot snp);
}
   /** */
   private static final ContextAttribute<String> ATTR = ContextAttribute.newInstance("initial-value");
 
   /** */
   public static void main(String[] args) {
       ContextSnapshot snapshot;
 
       try (Scope scope = Context.set(ATTR, "test")) {
           String attrVal = Context.get(ATTR);
       }
 
       ContextSnapshot snapshot = Context.createSnapshot();
 
       try (Scope scope = Context.restoreSnapshot(snapshot)) {
           String attrVal = Context.get(ATTR);
       }
   }


Risks and Assumptions

The proposed improvement allows to largely eliminate the problem of losing the context of an operation in a multi-threaded environment, but, unfortunately, does not solve this problem completely.
External libraries may provide the ability to submit code for multithreaded execution, but obviously do not support context propagation. Like JDK mechanism - java.util.stream.BaseStream#parallel. Misuse of such mechanisms can still result in loss of operation context and should be treated with caution.

Ubiquitous support for context capture and recovery may impact performance and GC. Although preliminary tests did not reveal any noticeable impact.

Discussion Links

// Links to discussions on the devlist, if applicable.

Reference Links

https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/ScopedValue.html

https://github.com/grpc/grpc-java/blob/master/api/src/context/java/io/grpc/Context.java

https://github.com/open-telemetry/opentelemetry-java/tree/main/context/src/main/java/io/opentelemetry/context

https://projectreactor.io/docs/core/release/reference/advanched-contextPropagation.html

https://github.com/micrometer-metrics/context-propagation

https://logback.qos.ch/manual/mdc.html

Tickets

Key Summary T Created Updated Due Assignee Reporter P Status Resolution
Loading...
Refresh

  • No labels