THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
Status
...
| Page properties | |
|---|---|
|
...
...
| Jira | ||||||
|---|---|---|---|---|---|---|
|
|
...
|
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
We will split this JIRA into two 3 tasks.
The first task: Ignore max buffers per channel when allocate buffer
...
Adding the new configuration : taskmanager.network.memory.max-overdraft-buffers-per-gate, the default value is 5~10?5.
The LocalBufferPool will be unavailable when the maxBuffersPerChannel is reached for this channel or availableMemorySegments.isEmpty or numberOfRequestedOverdraftMemorySegments > 0.
...
The third task: Compatible with LegacySource
| Jira | ||||||
|---|---|---|---|---|---|---|
|
Since LegacySource does not have checkAvailable, LegacySource will use all overdraft buffers by default, this is not what we expected.
So we'll detect the LegacySource and set overdraft=0 on the Task containing the LegacySourcefor the SourceStreamTask.
Compatibility, Deprecation, and Migration Plan
...
- Test for apply for overdraft buffer when overdraft buffer is sufficient
- Test for apply for overdraft buffer when overdraft buffer is insufficient
- Checkpoint Duration Benchmark for enable overdraft buffer
Rejected Alternatives
After discussing, we decided to use overdraft-buffer instead of reserve-buffer. For details, please refer to the mail list.