...

Motivation

Cache and Compute async operations invoke the future listeners on Ignite thread pools, such as Public pool and Striped pool:Striped pool threads, which can cause deadlocks and/or reduce cache performance.

IgniteFuture fut = cache.putAsync(1, 1);
fut.listen(f -> {
    // Executes on Striped pool and causes a deadlockdeadlocks.
    cache.replace(1, 2);
});

Users are supposed to be aware of this and handle it manually, however:

...

await cache.PutAsync(1, 1);
// Now we are on a Striped pool thread!

// CPU-heavy method blocks the stripe and cache ops are stalled.
RunSomething();

A similar problem exists for Compute. Async operation continuations are executed on the Public pool, which can lead to starvation there when all threads are taken up by continuation logic.

Description

• Add IgniteConfiguration#asyncContinuationExecutor (of type Executor).
• Use ForkJoinPool#commonPool by default (when null / not set).
• Use this executor for all Cache and Compute async continuations

This fixes the issue in Java, .NET and C++, because thick integrations use direct JNI callbacks for Futures.

NOTE: This IEP is NOT related to scan query filters, cache entry processors, etc, which also run on Striped pool.

...

Those users can force the old behavior with `IgniteConfiguration.setAsyncContinuationExecutor(Runnable::run)`.

Performance

Executing continuation on a different thread involves some overhead. Local benchmark with integer key and value shows ~6% drop (see JmhCacheAsyncListenBenchmark in the PoC).

In a real world workload the difference should be insignificant.

Benchmark                          Mode  Cnt      Score      Error  Units
JmhCacheAsyncListenBenchmark.put  thrpt   10  77859.584 ± 2071.196  ops/s (before)
JmhCacheAsyncListenBenchmark.put  thrpt   10  73393.986 ± 1336.420  ops/s (after)

Discussion Links

IEP thread: http:// Links to discussions on the devlist, if applicable./apache-ignite-developers.2346864.n4.nabble.com/IEP-70-Async-Continuation-Executor-td51775.html

Original discussions: 

• http://apache-ignite-users.70518.x6.nabble.com/Replace-or-Put-after-PutAsync-causes-Ignite-to-hang-td27871.html
• http://apache-ignite-developers.2346864.n4.nabble.com/Re-EXTERNAL-Re-Replace-or-Put-after-PutAsync-causes-Ignite-to-hang-td42921.html

Reference Links

PoC: https://github.com/apache/ignite/pull/8870

...