...
Numbers are based on the items in the list below, not on the tickets
∑ | Remaining weeks | |||||||||
2022-11-01 | 13 | |||||||||
2022-11-15 | 11 | |||||||||
2022-11-29 | 4 | 10 | 0 | 0 | 0 | 23 | 0 | 37 | 9 | |
2022-12-13 | 7 | 24 | 1 | 0 | 1 | 9 | 0 | 42 | 7 | |
2022-12-27 | 7 | 25 | 1 | 0 | 1 | 8 | 0 | 42 | 5 | |
2023-01-10 | 9 | 27 | 0 | 0 | 3 | 5 | 0 | 44 | 3 | |
2023-01-17 | 13 | 23 | 0 | 0 | 3 | 3 | 2 | 44 | 2 | |
2023-01-24 | 16 | 21 | 0 | 0 | 3 | 3 | 2 | 45 | 1 | |
2023-01-31 | 2726 | 5 | 0 | 0 | 1516 | 0 | 0 | 47 | 0 | |
2023-02-14 | 5 | 23 | 3 |
List
Feel free to add categories.
Runtime | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||||||||
| FLIP in voting | |||||||||||||||||
Shuffle | ||||||||||||||||||
| Xintong Song |
100% | 10-01-2023 | 10-01-2023 |
14-02-2023 |
| |||||||||||||||||
AdaptiveBatchScheduler should supports early consumption for dynamic graph. | Weijie Guo | Xintong Song | ||||||||||||||||
| Yuxin Tan | Xintong Song | 100% | 17-01-2023 | 17-01-2023 | 10-02-2023 | self-test | |||||||||||
Further improvement of production availability of hybrid shuffle |
100% | 17-01-2023 | 17-01-2023 | self-test | ||||||||||||||
Deployment & Cluster Coordination | |||||||||||||||||
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
REST API | |||||||||||||||||
|
100% |
20- |
02- |
2023 | 15-11-2022 | 15-11-2022 | self-test | |||||||||||||||||||||||
| The first stage is finished, and the second stage is improvement, it will be finished in the next version. | 100% | 09-01-2023 | 05-01-2023 | 09-01-2023 | |||||||||||||||||||||
Reactive Mode | ||||||||||||||||||||||||||
n/a | ||||||||||||||||||||||||||
Scheduler | ||||||||||||||||||||||||||
| 100% | 17-01-2022 | ||||||||||||||||||||||||
| Biao Liu | 100% | 31-01-2022 |
| ||||||||||||||||||||||
| 100% | 31-01-2023 | ||||||||||||||||||||||||
Misc | ||||||||||||||||||||||||||
n/a | ||||||||||||||||||||||||||
OLAP | ||||||||||||||||||||||||||
n/a | ||||||||||||||||||||||||||
State backend | ||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||||||||||||||||
FLIP-263: Improve resolving schema compatibility
| Hangxiang Yu | 80% | 31-1-2023 | |||||||||||||||||||||||
Improve File Management in State Backend | 5% | 31-1-2023 | ||||||||||||||||||||||||
Improve the serializer performace of state change of changelog
|
100% | 31-1-2023 | self-test | ||||||||||||||||||||
Allow to configure Changelog Storage per program
| 70% | 31-1-2023 | ||||||||||||||||||||
Add a metric for back-pressure from the ChangelogStateBackend
| 40% | 31-1-2023 | ||||||||||||||||||||
Frocksdb cannot run on Apple M1
|
100% | 30-1-2023 | Tested by Martijn Visser | ||||||||||||
Release FRocksDB 6.20.3-ververica-2.0
|
100% | 31-1-2023 | Tested by Martijn Visser | ||||||||||||||||||||
Checkpoint | ||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||||||||||||
Benchmark Generic Incremental CP + UC + BD | Yuan Mei | |||||||||||||||||||||
| 100% |
| ||||||||||||||||||||
Benchmark | ||||||||||||||||||||||
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||||||||||||
Improve benchmark stability
| Yuan Mei | 50% | 31-1-2023 | |||||||||||||||||||
API | ||||||||||||||||||||||
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||||||||||||
| Deprecated |
100% | 08-11-2022 | 08-11-2022 | 08-11-2022 | Self-tested | ||||||||||||||
| Removed | 40% | 31-01-2023 | |||||||||||||||
| Dawid Wysakowicz | 90% | 31-01-2022 | |||||||||||||||
| FLIP Discussion | Need to be postponed to the next release | 10-01-2023 | |||||||||||||||
| Martijn Visser | Removed |
100% | 14-11-2022 | 14-11-2022 | 14-11-2022 | Self-tested | ||||
|
90% | 31-01-2022 | Self-tested | ||||||||||||||||
| Ruan Hang | Coding | ||||||||||||||||
SQL | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||||||||
General | ||||||||||||||||||
| Godfrey He ,@Yunhong Zheng | 100% | 2023-01-31 | |||||||||||||||
| 31-01-2023 | |||||||||||||||||
| Chesnay Schepler | 31-01-2023 | ||||||||||||||||
Table API | ||||||||||||||||||
|
2023-01-20 | ||||||||||||||||||
Calcite Update | ||||||||||||||||||
| ||||||||||||||||||
| ||||||||||||||||||
| Sergey Nuyanzin | 31- |
01-2023 | ||||||||||
Flink Dialect | ||||||||||
ALTER TABLE API
|
100% | 2023 |
-02-14 | self-tested | ||||||||
|
100% |
2023- |
02-14 | self-tested | |||||||||||||||||||||||||
Hive Dialect | ||||||||||||||||||||||||||
|
|
|
| 100% | 2023-01-31 | ||||||||||||||||||||||||
| 31-01-2022 | |||||||||||||||||||||||||
SQL Gateway | ||||||||||||||||||||||||||
| Shengkai Fang , Zelin Yu | writing doc | 100% | 2023-01-10 |
| |||||||||||||||||||||
QE | ||||||||||||||||||||||||||
|
100% | 2023-01-10 |
| ||||||||||||||||
QO | ||||||||||||||||||
FLINK-27591 - Improve the plan for batch queries when statistics is unavailable OPEN | Godfrey He ,@Yunhong Zheng | Need to be postponed to the next release | 40% | 2023-1-31 | ||||||||||||||
API/Python | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||||||||
| HuangXingbo | 100% | 10-01-2023 | self-test | ||||||||||||||
| HuangXingbo | 0% | 10-01-2023 | |||||||||||||||
| HuangXingbo | 100% | 19-01-2023 | self-test | ||||||||||||||
| HuangXingbo | 100% | 19-01-2023 | self-test | ||||||||||||||
| HuangXingbo | 100% | 10-01-2023 | self-test | ||||||||||||||
Machine Learning | ||||||||||||||||||
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||||||||
n/a | ||||||||||||||||||
CEP | ||||||||||||||||||
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||||||||
n/a | ||||||||||||||||||
Web | ||||||||||||||||||
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||||||||
| 100% |
Status / Follow-ups
2022-11-01
...
- Today is feature freeze day
- 27 26 features / improvements are in for Flink 1.17 (47 in at Flink 1.16, 20 for Flink 1.15, 27 for Flink 1.14)
- 5 features are still listed as expected to be completed, but are not yet in, 4 of them have been merged and are writing documentation, Martijn Visser to check/update for the status of these items.
- Blockers:
Jira server ASF JIRA
→ Matthias will look into thiscolumnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,customfield_12311032,customfield_12311037,customfield_12311022,customfield_12311027,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,Priority,Priority,Priority,Priority,priority,status,resolution serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30881 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-
30727
→29427
PR to update buffers for the test has been merged, will be continued to monitor. Test downgraded to Critical- Leonard review PASS, waiting CI green
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-
30625 → Should be resolved, pending validation by the benchmarks (related to FLINK-30624). Benchmarks look to be improved, downgraded to Critical.
→ Fixed by Qingsheng29405 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-
29427
→30625
Qingsheng has tested this in own CI, looks to be resolved and he'll create a PR for this shortly- Should be resolved, pending validation by the benchmarks (related to FLINK-30624). Benchmarks look to be improved, downgraded to Critical.
→ Should be resolved, pending validation by the benchmarks (related to FLINK-30625). Benchmarks look to be improved, downgraded to Critical.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30624
→ Martijn to reach out to Dong, Rui Fan, Piotr. We're planning to give them until Friday the 3rd of February to come to a conclusion on this ticket; if no consensus is achieved, then the original commit that introduced the regression should be reverted.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30623
→ Matthias to check if this has already been resolved via another ticket (multiple related tickets)Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30826 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-
30846
→30727
Qingsheng will check with Zhu Zhu if this is related to recent changes- [Critical] PR to update buffers for the test has been merged, will be continued to monitor. Test downgraded to Critical
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-
30844
→30846
Matthias to look into this test
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b → PR open, Qingsheng will take a final pass on thiskey FLINK-29405 - [Critical] Downgraded to Critical as it's only a test-related issue and doesn't indicate a bug in production.
→ [Major] Test downgraded to Major, If the test fail again contributor will increase the waiting interval for this test as a solution.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30844
→ [Major] Leonard downgrade the issue priority to Major as it's a known slack plugin issueJira server ASF JIRA columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,customfield_12311032,customfield_12311037,customfield_12311022,customfield_12311027,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,Priority,Priority,Priority,Priority,priority,status,resolution serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30870
- Martijn Visser to communicate to the Flink community this evening that the feature freeze has started and we plan to cut the release branch at the end of this week (Friday 3rd of February).
2023-02-07
master
is stabilized enough to cut therelease-1.17
branch- FLINK-30921 - The Azure apt mirror instabilities seem to have been resolved for now.
- FLINK-30908 - The issue turned out to be a problem that existed in previous releases
- FLINK-30921 - The Azure apt mirror instabilities seem to have been resolved for now.
- Release branch is going to be cut today by Leonard Xu
- Release testing will be announced: 2 weeks will be planned for this
2023-02-14
- Status update on release testing efforts
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30926 - End of cross-team testing date is 21st of February 2023. We will monitor the status throughout the week and hopefully conclude everything next week.
- When a feature is cross-team tested, the icon needs to be changed from to to indicate that the testing has been completed
- We always look for volunteers: picking up a cross-team testing task is much appreciated.
- Proposal: Create Jira issues for release management tasks to document what was done to improve review-ability (alternatively, add expected output to release documentation)
- Do this from now on for the next steps in release management (create release candidate etc.)
- Test instabilities:
: BlockerJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31036
: Starts failing more regularly again. Qingsheng Ren to ping Godfrey He and Yunhong ZhengJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18356 - FLINK-30972 (openssl version update necessary) continues to fail because ofFLINK-30965 (repo-sync doesn't pick up 1.15 changes anymore)
2023-02-21
- Open Blocker issues
- Example Release Jira issues:
>Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31146 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31154 - Release testing not finished, yet
- Decision on RC creation moved to next week's Flink release sync call
- Other issues:
: Issue in 1.15.3 unexplainedJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31133
: 1.17 issue in HiveJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31092
: no updates on the Slack bot instability so farJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30733
: OOM issue - waiting for Gabor's responseJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31134
(Kafka infrastructure umbrella ticket)Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31145
2023-02-28
- Blockers: are these really blockers or do we want to move the release forward?
→ Qingsheng Ren to check the status with Shengkai FangJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31092
→ Qingsheng Ren to check the status with Shengkai FangJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31104
→ Qingsheng Ren to check the status with Shengkai FangJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30978
- Other issues:
: Revisit OOM in Kafka e2e test in 1.15.3 - we're keeping it as critical for now. There are no other artifacts that we can investigate. The test doesn't involve table-runner code (therefore, isn't connected toJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31134
). → Martijn Visser to check internallyJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18356 - Dependabot alerts are affecting all commiters now
- Release managers to review draft of 1.17 announcement
- Release 1.17 preparation:
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31146
2023-03-07
- Ubuntu mirror instabilities:
Matthias Pohl will follow-up on that oneJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30921 - API backwards compatibility:
Do we have additional documentation on API compatibility? Leonard Xu will do another pass over the ticket comments to confirm the findingsJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31167 - Announcement: FLIPs missing: FLIP-272: Generalized delegation token support, FLIP-217: Support watermark alignment of source splits Leonard Xu reaches out to contributors to add something related to the announcement
- Instabilities (blocker query):
:Leonard Xu has pinged luoyuxia to look into this issue.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31351
: Downgraded→ [Critical] PR ready, flink-runtime tests will be made sequential ,Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31278
: Downgraded→ [Critical] Leonard Xu has pinged Godfrey He to look into this issue.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31339
: Downgraded→ [Critical]Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31342
: Downgraded→ [Critical]Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31341
: Downgraded→ [Critical] Matthias PohlJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31354
: Downgraded→ [Critical] Leonard Xu has pinged Dian Fu to look into this issue.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31355
2023-03-14
- Flink RC2 for 1.17 is available, needs checkers
2023-03-21
- RC3 for 1.17.0 was created and is in voting stage (1 binding vote miss)
- Robert Metzger agreed to assist with the license check
2023-03-23
- Flink 1.17.0 is officially released today!
Retrospective:
From Qingsheng Ren
- As discussed in the mailing list, we need to trigger a final patch version for 1.15 after releasing 1.17. Some cleanup steps need to be reviewed and changed, such as removing 1.15 data from svn, CI, flink-docker etc. See
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31570 - I like the idea made by Matthias Pohl that we track TODOs for releasing on JIRA 👍 I used it as a checklist to make sure we don't miss anything. Also it helps collaborating, as we can divide works across RMs easily by assigning JIRA tickets.
From Matthias Pohl
Google Meet might not be the best choice for the release sync. We need to be able to invite attendees even if the creator of the meeting isn't available (maybe try Zoom or even Jitsi as an OpenSource alternative instead?)
- Release sync every 2 weeks and a switch to weekly after feature freeze felt reasonable
Slack worked well as a collaboration tool to document the monitoring tasks (#builds, #flink-dev-benchmarks) in a team with multiple release managers
The Slack Azure Pipeline bot seems to be buggy. It swallows some build failures. It's not a severe issue, though. We created #builds-debug to monitor whether it's happening consistently. The issue is covered in
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30733 We experienced occasional issues in the manual steps of the release creation in the past (e.g. japicmp config was not properly pushed). Creating Jira issues for the release helped to make the release creation more transparent and made the steps more reviewable. Additionally, it helped to distribute subtasks to different people with Jira being the tool for documentation and synchronization. That's especially helpful when there is more than one person in charge of creating the release.
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31146 - RCs
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31154 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31578 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31583
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31562 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31567
We had backports/merges without PRs happening by committers occasionally during the 1.17 release which broke master/release branches (probably, changes were done locally before merging which were not part of the PR to have a faster backport experience). It might make sense to remind everyone that this should be avoided. Not sure whether we want/can restrict that.
We observed a good response on fixing test instabilities by the end of the release cycle but had some long running issues earlier in the cycle which caused extra efforts on the release managers due to reoccurring test failures.
Release testing picked up “slowly”: Initially, we planned 2 weeks for release testing. But there was not really any progress (tickets being created and worked on) in the first week. In the end, we had to extend the phase by another week resulting in 3 instead of 2 weeks of release testing. I guess we could encourage the community to create release testing tasks earlier and label them properly to be able to monitor the effort. That would even enable us to do release testing for a certain feature after the feature is done and not necessarily only at the end of the release cycle.
- Manual test data generation is tedious (
). But this should be fixed in 1.18 withJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31593
being almost done.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-27518 - We started creating documentation for release management. The goal is to collect what tasks are there to help support a Flink release to encourage newcomers to pick up the task.
From Leonard Xu
- We can keep RC0 (a non-votable one) in future releases, as an initial version for developers to validate, so that some issues could be found earlier and avoid repeatedly canceling and re-creating RCs.
From Martijn Visser
- We should be more careful for commits without a PR / green CI, which brought some problem at the end of 1.17 release cycle. There might not be possible to totally ban this, but we could give an reminder to committers.