...
Currently Camel uses thread pools in camel-core in the following areas:
DefaultComponent
- Optional thread pool for components in need of suchDefaultEndpoint
- Optional thread pool for endpoints in need of suchDefaultProducerTemplate
- Used by the Async API of this templateScheduledPollConsumer
- Needs aScheduledExecutorService
to schedule its tasksRecipientListDefinition
- The Recipient List EIP patternSplitDefinition
- The Splitter EIP patternThreadsDefinition
- Thethreads
DSLToDefinition
- Used by thetoAsync
variationWireTapDefinition
- The Wire Tap EIP patternMulticastProcessor
- The underlying thread poolOnCompletionProcessor
- For sending async on completion routesSendAsyncProcessor
- ThetoAsync
processor variationThreadsProcessor
- The underlying thread poolWireTapProcessor
- For sending async wire tapsSedaConsumer
- To support theconcurrentConsumers
andmultipleConsumers
options (uses separate pools)
Existing configuration
You can configure the thread pool using the setExecutorService setter methods that usually exists on those places where its in use. Some EIP patterns offer a executorServiceRef
option to refer to some pool to be looked up in the Registry.
...
Where it will match against route first, so if we got a route3 then it will pick among those
It should be possible to use wildcard and reg exp in the route
and source
attributes.
Status: Consider for the future
Default thread pool profile
It should be possible to set a default ThreadPoolProfile
which Camel will use when creating thread pools for the EIPs and whatnot. Then you can set default settings and have that leveraged out of the box.
Code Block | ||||
---|---|---|---|---|
| ||||
<threadPoolProfile id="myDefaultProfile"
defaultProfile="true"
poolSize="5" keepAliveTime="25" maxPoolSize="15" maxQueueSize="250" rejectedPolicy="Abort"/>
|
If none defined, then Camel should use a sensible default of
- poolSize = 10
- maxPoolSize = 20
- keepAliveTime = 60 seconds
- maxQueueSize = 1000
- rejectedPolicy = CallerRuns
And it should validate that only one defaultProfile=true
can be set.
Status: DONE
The problem with shutdown and restarting pools
...
We may have to only shutdown thread pools if CamelContext is stopping. And then if end user stop a route from JMX we can keep the thread pool around.
Only issue is the scheduled pool should stop scheduling tasks, which may be a bit more trickier to avoid.
We have introduced a ShutdownableService
to expose a shutdown
method which services can implement for their shutdown logic.
Then we can only shutdown thread pools in doShutdown()
and not as before in doStop()
.
By letting Camel keep track of created thread pool, then Camel knows which pools shutdown when its stopping. Then the need for doShutdown
is not as apparent as before, but its good to have this state in the lifecycles as well, for future needs.
Status: DONE
The problem with Component, Endpoint
...
Check whether the thread pools is managed by default and avail in JConsole. If not we should probably at least expose some read-only data about the pools. DONE
Spring Factory for creating custom pools
...
We should offer a simple pattern syntax so end users can customize the pattern the thread name is created with: eg Something like: Camel Thread ${counter} - ${name
}. Where counter and suffix is dynamic parameters. The counter is an unique incrementing thread counter. And name is provided from the source which as a way to name thread, such as a seda endpoint uri. DONE
EIP should mandate an ExecutorService
If the EIPS which leverages a ExecutorService, mandates its being created and passed to it, we can enforce creating/lookup the pool during route creation, which allows us to have the route information as well, so we know which routes creates which pools. By passing in null
we loose this opportunity.
That is why all the EIP Processors should be refactored to have ExeuctorService as parameter. DONE
Let Camel keep track of created pools
Using the DefaultExecutorServiceStrategy
we can let Camel keep track of the created pools, and thus also it can shutdown those when CamelContext is shutting down. Then Camel is handling the lifecycle for the pools it creates. And if you pass in a thread pool from an external system then you manage that lifecycle. Camel will in those cases not shut it down. DONE
Sensible defaults
The CachedExecutorService
by the JDK is maybe a bit aggressive as its unbounded thread pool which essentially can create 1000s of threads if the sever is not busy. But end users may want to have a reasonable max size, lets say 100. So we should offer some sort of rule which you can configure what the default settings should be for thread pools created by Camel. DONE
Rejection policy
We should add configuration about rejection policies for new tasks submitted to a pool. The JDK has options for ABORT, RUN, WAIT, DISCARD etc. DONE