THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
Status
...
| Page properties | |
|---|---|
|
...
...
|
...
JIRA:
...
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
The user specifies the customized parallelism through connector options:
Option | Type | Default value |
sink.parallelism | Integer | None - Chained: Use upstream parallelism - Non-Chained: Use global parallelism setting |
SourceProvider
ScanRuntimeProvider for FLIP-27:
...
public interface SourceProvider extends ScanTableSource.ScanRuntimeProvider, ParallelismProvider {
/**
* Helper method for creating a static provider.
...
The user specifies the customized parallelism through connector options:
Option | Type | Default value |
scan.parallelism | Integer | None (Use global parallelism setting) |
Infer Scan parallelism
Connector | Can infer Source or Sink | How to infer |
Kafka | Unbounded Source | Infer by partitions |
Filesystem / Hive / Iceberg | Bounded Source | Infer by split numbers |
JDBC/HBase | Bounded Source | Infer by split numbers |
Elasticsearch | None |
As can be seen, most connectors infer parallelism according to split numbers, and only infer source parallelism. But it can't rule out that users have customized parallelism inference mode.
User can control inference logical by connector options:
Option | Type | Default value |
scan.infer-parallelism.enabled | Boolean | True |
scan.infer-parallelism.max | Integer | None (Use global parallelism setting) |
(The global parallelism setting is StreamExecutionEnvironment.getParallelism, in table, it can be configured by “table.exec.resource.default-parallelism”)
...