...
- The release sync will start from Nov 14th, 2023, at 9am (CET, UTC+1) and 4pm (UTC+8). The release sync happens bi-weekly at first, and will be adjusted to weekly as we approaching the feature freeze date.
- Feel free to join on Google Meet, and attendees are encouraged to fill out the topics to be discussed at the bottom of this page a day in advance to make it easier for everyone to understand the background of the topic.
Timeline
- Feature Freeze
- Jan 26, 2024, 00:00 CET(UTC+1)
- Release
- End of Feb 2024
...
Numbers are based on the items in the list below, not on the tickets
∑ | Remaining weeks | |||||||||
2023-11-28 | 2 | 11 | 21 | 9 | ||||||
2023-12-12 | 3 | 11 | 23 | 7 | ||||||
2023-12-26 | 4 | 15 | 27 | 5 | ||||||
2024-01-09 | 4 | 15 | 1 | 27 | 3 | |||||
2024-01-23 | 15 | 7 | 2 | 1 | 4 | 29 | 1 | |||
2024-01-30 | 24 | 0 | 2 | 1 | 4 | 31 | 0 | |||
2024-02-06 |
List
Feel free to add categories.
Runtime | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | Volunteers for testing |
Shuffle | |||||||||||
Coordination | |||||||||||
FLIP-379: Dynamic source parallelism inference for batch jobs |
2024- |
01- |
23 | ||||||||||
|
Accepted |
2024-01-22 | |||||||||||||||
| Done | All features are merge, it only left a little test and doc, so marking it done first. | 2024-01-22 |
FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces |
Done | 2024-01-18 | ||||||||
FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter | Piotr Nowojski |
Done | 2024-01-24 | ||||||||||
Done | 2024-01-24 | ||||||||||
FLIP-383: Support Job Recovery for Batch Jobs |
Accepted | 2024-01-22 |
State & Checkpoint | |||||||||||
|
| 2024-1-15 | ||||||||||||||||||
| 2024-1-24 | ||||||||||||||||||
REST | |||||||||||||||||||
Configuration | |||||||||||||||||||
| 2024-01-26 | ||||||||||||||
FLINK-33712 - FLIP-391: Deprecate RuntimeContext#getExecutionConfig | 2024-01-16 | ||||||||||||||
FLINK-33581 - FLIP-381: Deprecate configuration getters/setters that return/set complex Java objects | 2024-01-16 | ||||||||||||||
| Done | 2024-01-22 |
SQL | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | |
|
2024-01-30 | |||||||||||||||||||||||
| @Yangze Guo | ||||||||||||||||||||||
FLIP-376: Add DISTRIBUTED BY clause | 2024-01-30 | ||||||||||||||||||||||
no new flip needed, continue the work on FLINK-24024 & FLINK-20281 | 2024-01-26 | FLINK-20281: N/A | |||||||||||||||||||||
FLIP-393: Make QueryOperations SQL serializable | Done | 2024-01-18 | N/A | ||||||||||||||||||||
FLIP-387: Support named parameters for functions and call procedures | @Feng Jin | xuyang | need documentation | 2024-01-30 | |||||||||||||||||||
| Under discussion | 2024-01-18 | |||||||||||||||||||||
need documentation | 2024-01-30 | ||||||||||||||||||||||
Timo Walther |
Needs documentation | 2024-02-05 | ||||||||||||||||||
Connectors | |||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2024-01-22 | ||||||||||||||||||
| Pulsar connector has formal release already | 2024-01-23 | |||||||||||||||||
|
only left for changes in kafka connector | 2024-01-22 | |||||||||
FLIP-371: Provide initialization context for Committer creation in TwoPhaseCommittingSink |
Done | 2024-01-18 | |||||||||||
|
FLIP-372: |
Enhance and synchronize Sink API to match the Source API |
| Mark it done since main part of the new api been merged (remaining tests and doc) | 2024-01-23 | |||||||||||||||||||||
| |||||||||||||||||||||||
Resource Management | |||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Runtime | |||||||||||||||||||||||
| Rui Fan | RocMarshal |
Done | 2023-12-04 | ||||||||||||
A series of optimizations to reduce the memory and cpu usage for large StreamOperator | Rui Fan |
2023-11-23 | |||||||||||
Misc |
|
DONE, only docs left | 2024-01-23 | ||||||||
|
|
(will create a FLIP) |
2024-01-23 |
Status / Follow-ups
11/14/2023
- Is Google Meet OK for everyone? - ok
- The release sync meeting time, currently Europe has gone into winter time(CET, UTC+1), is 9am ok? - ok
- Daily work divisions
- JIRA ticket (Mainly focus on new blockers, progress of existing blockers each week) lincoln lee
- CI (Failures reported in #build channel and not reported) Martijn Visser
- Benchmark (Check if there's regression in the #flink-dev-benchmarks channel) Yun Tang
- Mailing list monitoring Jing Ge
- This division doesn't mean that other RMs are now allowed to do the work. All RMs should take responsibilities on all of these daily issues if they are available.
- Feature freezing date. - Jan 26, 2024, 00:00 CET(UTC+1)
- Retrospective of 1.18
11/28/2023
- Blockers
discuss thread - back to ml discussionJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-25857 - documentation build failures
- closedJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-33544
- GitHub Actions Migration discussion in dev ML (FLIP-396)
- flink-shaded driven by Sergey Nuyanzin
- As suggested by Xintong Song, we encourage attendees to fill out the topics a day in advance to make it easier for everyone to understand the background of the topic.
12/12/2023
- Current status of 1.19 Features lincoln lee - updated
- Existing blocker issues lincoln lee
discuss thread - closedJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-25857
(potential blocker)Jira server ASF JIRA columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,customfield_12311032,customfield_12311037,customfield_12311022,customfield_12311027,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,Priority,Priority,Priority,Priority,priority,status,resolution serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-33641
12/26/2023
- Current status of 1.19 Features lincoln lee - updated
- 5 weeks remaining, total 27, done 4, will make it 15
- Existing blocker issues lincoln lee
has been fixedJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-33641
- Skip this sync meeting? - no response until 12/25, skipped finally
01/09/2024
- Current status of 1.19 Features lincoln lee - updated
- 3 weeks remaining, total 27, done 4, will make it 15, won't make it 1
- Existing blocker issues lincoln lee
sergy reported a breaking change, will be fixed soon Wencong LiuJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-32978
01/23/2024
- Current status of 1.19 Features lincoln lee
- Jan 26th is the planned feature freeze day, total 28, done 15, will make it 6, won't make it 4;
- Existing blocker issues lincoln lee
Sergey Nuyanzin Zakelly LanJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34148
Matthias PohlJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34007
the disk-full issue was addressed, should be stable for nowJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34135 - Weird build on Friday (Jan 19): https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=56601&view=results (probably caused by infrastructure issues; but AFAIU, they were resolved the day before?)
- Whether to delay feature freeze day? Yun Tang
- Build pipeline stability
: python stage reaches 4h timeoutJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34202
:Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31472 AsyncSinkWriterThrottlingTest
quite unstable recently
: macos wheels build seems to be unstable (multiple open issues)Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34069
/Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34200
: new featureJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-33919
01/30/2024
- Current status of 1.19 Features lincoln lee
- We extend the feature freeze day to Jan 30th, total 31, done 24;
- FLIP-376: Add DISTRIBUTED BY clause could be done this week: do we want to wait, or do we want to cut the release branch? Martijn Visser → Will be postponed to 1.20
- Existing blocker issues lincoln lee
Sergey Nuyanzin Zakelly LanJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34148
Matthias PohlJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34007
&Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34225
@yunfengzhouJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34233 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34272
- When to cut release-1.19 branch?
- Cross-team release testing work
02/06/2024
- Blockers:
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34148 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34007
&Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34225 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34233 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34272
pr reviewing jiabao.sunJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34337
not yet clarified whether it's a production code or test issueJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-33958
- Cut release-1.19 branch
- fyi:
GitHub Actions e2e tests seem to have disk space issues causing multiple test failure (not present in Azure)Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34360 - Cross-team release testing
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34285 - total 26 flip/features, 15 confirmed, 7 not required, 4 wait for response(authors: Dawid Wysakowicz, Peter Vary, Alan Sheinberg)
- testing instructions ready: 11 (1 completed, 3 assigned, 7 unassigned), wait for doc ready: 2
- more completed flips found, will update the testing ticket (done, 6 Instructions tickets added)
02/13/2024
- Not verified Jira issues:
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34404 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34424
Matthias PohlJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34425
Matthias PohlJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34427
- CI instability issues:
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34273 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34202 - Nightly build for 1.19 doesn't seem to be picked
- Cross-team release testing
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34285 - wait for response: there's some authentication problem in current kanban dashboard, use filter on Jira search 'project = FLINK AND type = Sub-task AND parent = FLINK-34285 AND labels is EMPTY AND status = OPEN ORDER BY updatedDate'
- wait for doc ready:
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34310 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34305
- testing tickets: Jira filter condition 'project = FLINK AND type = Sub-task AND parent = FLINK-34285 AND labels = release-testing ORDER BY updatedDate'
02/20/2024
- Not verified Jira issues:
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34404
Yunfeng ZhouJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34424
Matthias PohlJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34425
Matthias PohlJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34427
- CI instability issues:
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34273
@Xingbo HuangJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34202
- Cross-team release testing
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34285 wait for response: there's some authentication problem in current kanban dashboard, use filter on Jira search 'project = FLINK AND type = Sub-task AND parent = FLINK-34285 AND labels is EMPTY AND status = OPEN ORDER BY updatedDate'- wait for doc ready:
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-34305
- testing tickets: Jira filter condition 'project = FLINK AND type = Sub-task AND parent = FLINK-34285 AND labels = release-testing ORDER BY updatedDate'
- 15 total, 8 closed, 4 unassigned, 3 testing
- When to create rc? –- after release testing finished and no blockers