...
Feel free to join on Google Meet. Local dial-in numbers can be found at https://tel.meet/wcx-fjbt-hhz?pin=1940846765126
Timeline
- Feature Freeze
January, 17th of 2023, end of business CEST.
...
- January 31st, 2023, end of business CEST (ML discussion on change)
- Release
Mid of March 2023- End of March 2023
Chinese new year is 22 January: 1+ week holiday. Ppl will be back in early February
Highlight features
please feel free to add/suggest.
...
symbol | meaning | comment |
---|---|---|
validated | through cross team testing | |
done | well documented with a complete test coverage | |
will make it | there is no reason this effort should not go into 1.17 | |
in danger | there are some concerns the effort could be ready for the feature freeze of 1.17 | |
very unlikely | there are severe concerns the effort could make it to 1.17 | |
won't make it | it was decided against adding this for the 1.17 release. working on the effort has been stopped. | |
state unclear | ||
independent | as the artefact artifact could be released independent of Apache Flink |
...
Numbers are based on the items in the list below, not on the tickets
∑ | Remaining weeks | |||||||||
2022-11-01 | 1113 | |||||||||
2022-11-15 | 0911 | |||||||||
2022-11-29 | 074 |
List
Feel free to add categories.
10 | 0 | 0 | 0 | 23 | 0 | 37 | 9 | |||
2022-12-13 | 7 | 24 | 1 | 0 | 1 | 9 | 0 | 42 | 7 | |
2022-12-27 | 7 | 25 | 1 | 0 | 1 | 8 | 0 | 42 | 5 | |
2023-01-10 | 9 | 27 | 0 | 0 | 3 | 5 | 0 | 44 | 3 | |
2023-01-17 | 13 | 23 | 0 | 0 | 3 | 3 | 2 | 44 | 2 | |
2023-01-24 | 16 | 21 | 0 | 0 | 3 | 3 | 2 | 45 | 1 | |
2023-01-31 | 26 | 5 | 0 | 0 | 16 | 0 | 0 | 47 | 0 | |
2023-02-14 | 5 | 23 | 3 |
List
Feel free to add categories.
Runtime | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified |
Jira | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
Jira | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
|
|
FLIP in voting | |||||||||||||||||||||
Shuffle | |||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Xintong Song | 100% | 10-01-2023 | 10-01-2023 | 14-02-2023 |
|
Allow to configure Changelog Storage per program
| ||||||||||
AdaptiveBatchScheduler should supports early consumption for dynamic graph. | Weijie Guo | Xintong Song | ||||||||
|
|
Add a metric for back-pressure from the ChangelogStateBackend
| Yuxin Tan | Xintong Song | 100% | 17-01-2023 | 17-01-2023 | 10-02-2023 | self-test | |||||
Further improvement of production availability of hybrid shuffle | 100% | 17-01-2023 | 17-01-2023 | self-test | ||||||||
Deployment & Cluster Coordination | ||||||||||||
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||
---|---|---|---|---|---|---|---|---|---|---|---|---|
REST API | ||||||||||||
|
Add a metric for back-pressure from the ChangelogStateBackend
|
|
Improve benchmark stability
100% | 20-02-2023 | 15-11-2022 | 15-11-2022 | self-test | ||||||||||||||
| The first stage is finished, and the second stage is improvement, it will be finished in the next version. | 100% | 09-01-2023 | 05-01-2023 | 09-01-2023 | |||||||||||||
Reactive Mode | ||||||||||||||||||
n/a | ||||||||||||||||||
Scheduler | ||||||||||||||||||
| 100% | 17-01-2022 |
|
| Biao Liu | 100% | 31-01-2022 |
|
|
| 100% | 31-01-2023 | ||||||||||||||||
Misc | ||||||||||||||||||
n/a | ||||||||||||||||||
OLAP | ||||||||||||||||||
n/a | ||||||||||||||||||
State backend | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available |
Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | |
---|---|---|---|---|---|---|---|---|
FLIP-263: Improve resolving schema compatibility
|
|
Hangxiang Yu |
80% |
Jira | ||||||
---|---|---|---|---|---|---|
|
31- |
1-2023 | ||||||||||
Improve File Management in State Backend | 5% | 31-1-2023 | ||||||||
Improve the serializer performace of state change of changelog
|
|
|
Jira | ||||||
---|---|---|---|---|---|---|
|
100% | 31-1-2023 | self-test | ||||||||||
Allow to configure Changelog Storage per program
|
| 70% | 31-1-2023 | ||||||||||
Add a metric for back-pressure from the ChangelogStateBackend |
|
| 40% | 31-1-2023 | ||||||||||
Frocksdb cannot run on Apple M1 |
|
|
Jira | ||||||
---|---|---|---|---|---|---|
|
100% | 30-1-2023 | Tested by Martijn Visser | |||||||
Release FRocksDB 6.20.3-ververica-2.0 |
|
|
|
|
Jira | ||||||
---|---|---|---|---|---|---|
|
100% | 31-1-2023 | Tested by Martijn Visser | ||||||||
Checkpoint | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified |
Benchmark Generic Incremental CP + UC + BD | Yuan Mei |
|
|
Shengkai Fang , Zelin Yu
FLINK-27591 - Improve the plan for batch queries when statistics is unavailable OPEN
100% |
| |||||||||
Benchmark |
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified |
---|
Jira | ||||||
---|---|---|---|---|---|---|
|
Improve benchmark stability
|
|
|
|
Yuan Mei |
Jira | ||||||
---|---|---|---|---|---|---|
|
50% | 31-1-2023 | |||||||||
API |
---|
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified |
---|
| Deprecated | 100% | 08-11-2022 | 08-11-2022 | 08-11-2022 | Self-tested |
|
Status / Follow-ups
2022-11-01
Agenda
...
| Removed | 40% | 31-01-2023 | ||||||||||||
| Dawid Wysakowicz | 90% | 31-01-2022 |
...
|
...
2022-11-15
Agenda
...
| FLIP Discussion | Need to be postponed to the next release | 10-01-2023 |
|
...
|
...
...
Removed |
...
...
100% |
...
14- |
...
11- |
...
2022 | 14- |
...
11- |
...
2022 | 14-11-2022 | Self-tested |
...
|
...
|
...
90% | 31-01-2022 | Self-tested |
...
|
...
|
...
Ruan Hang | Coding | |||||||||
SQL | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified |
General | ||||||||||
|
...
| Godfrey He ,@Yunhong Zheng | 100% | 2023-01-31 | |||||||||||||||||||||||||||
| 31-01-2023 | |||||||||||||||||||||||||||||
| Chesnay Schepler | 31-01-2023 | ||||||||||||||||||||||||||||
Table API | ||||||||||||||||||||||||||||||
| 2023-01-20 | |||||||||||||||||||||||||||||
Calcite Update | ||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||
| Sergey Nuyanzin | 31-01-2023 | ||||||||||||||||||||||||||||
Flink Dialect | ||||||||||||||||||||||||||||||
ALTER TABLE API
| 100% | 2023-02-14 | self-tested | |||||||||||||||||||||||||||
| 100% | 2023-02-14 | self-tested | |||||||||||||||||||||||||||
Hive Dialect | ||||||||||||||||||||||||||||||
|
| |||||||||||||||||||||||||||||
| 100% | 2023-01-31 | ||||||||||||||||||||||||||||
| 31-01-2022 | |||||||||||||||||||||||||||||
SQL Gateway | ||||||||||||||||||||||||||||||
| Shengkai Fang , Zelin Yu | writing doc | 100% | 2023-01-10 |
| |||||||||||||||||||||||||
QE | ||||||||||||||||||||||||||||||
| 100% | 2023-01-10 |
| |||||||||||||||||||||||||||
QO | ||||||||||||||||||||||||||||||
FLINK-27591 - Improve the plan for batch queries when statistics is unavailable OPEN | Godfrey He ,@Yunhong Zheng | Need to be postponed to the next release | 40% | 2023-1-31 | ||||||||||||||||||||||||||
API/Python | ||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||||||||||||||||||||
| HuangXingbo | 100% | 10-01-2023 | self-test | ||||||||||||||||||||||||||
| HuangXingbo | 0% | 10-01-2023 | |||||||||||||||||||||||||||
| HuangXingbo | 100% | 19-01-2023 | self-test | ||||||||||||||||||||||||||
| HuangXingbo | 100% | 19-01-2023 | self-test | ||||||||||||||||||||||||||
| HuangXingbo | 100% | 10-01-2023 | self-test | ||||||||||||||||||||||||||
Machine Learning | ||||||||||||||||||||||||||||||
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||||||||||||||||||||
n/a | ||||||||||||||||||||||||||||||
CEP | ||||||||||||||||||||||||||||||
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||||||||||||||||||||
n/a | ||||||||||||||||||||||||||||||
Web | ||||||||||||||||||||||||||||||
Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | ||||||||||||||||||||
| 100% |
Status / Follow-ups
2022-11-01
Agenda
- Kickoff
- Keeping the state of features updated (ideally before the sync)
- Blockers
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-24119
- Build stability
- Are there any (new) contributors who need a PR reviewed or merged? And if yes, who can help out?
2022-11-15
Agenda
- Kickoff
- Keeping the state of features updated (ideally before the sync)
- Blockers
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29387 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29315 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29818
- Build stability
- Number of growing test stability issues with "Exit code 137 errors"
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18356 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29899 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-26674
- Number of growing test stability issues with "Exit code 137 errors"
- Are there any (new) contributors who need a PR reviewed or merged? And if yes, who can help out?
2022-11-29
- Build instabilities
: Anton has some new findings on that issue and will get back on itJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-28766 - Pulsar-related issues:
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29835 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29755 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29836
- Python-related issues:
Matthias Pohl pings Xingbo Huang / Dian FuJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29461 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-26974
: OOM errors are most likely being caused byJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18356 flink-table-planner
(Godfrey He might be a person to reach out to about it)
: Qingsheng Ren will look into the PRJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29427
Matthias Pohl will ping contributors on the issue - but it's not that urgent since it's not failing that frequentlyJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-27916
- Other topics:
- Externalizing Pulsar connector (test instabilities): Martijn Visser is working on externalizing the code base
- Performance test monitoring: Discussion is happening on the mailing list
- Externalizing connectors in general is work-in-progress but looks good
- Public CI documentation can be improved
- Matthias Pohl will work on a first approval
2022-12-13
- Build instabilities
→ Qingsheng to have a look at the PRJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29405 - Pulsar connector has been synced to external connector repository. Martijn Visser to open a PR to remove connector from master
→ Qingsheng to ping GodfreyJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18356
→ The PR is still failing for the same issue as this PR should fix. We should ping the author to have a look firstJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18356
→ Martijn to ping Thomas once moreJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-27916
→ Xingbo is working on thisJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-26974
- How to have monitoring and quality control for the externalized connectors → Need to have a discussion on the Dev mailing list. Martijn Visser to make a proposal and open a discussion thread on this topic.
2022-12-27
- Meeting skipped due to Christmas holiday/sick leaves
- Discussion started on moving the feature freeze from Jan 17 to Jan 31 due to pandemic situation in China (see dev ML discussion thread)
- Pulsar connector has been externalized. Pulsar-related test instabilities were disabled in
release-1.16
andrelease-1.15
(see
and parent task)Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30351
2023-01-10
- Build instabilities (all 1.17 test instabilities with a priority >=Major)
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18356 - Godfrey He and Yunhong Zheng are working on it
- run tests after each other/not in parallel
- don't reuse JVMs
- JUnit has feature to log memory consumption
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-26974 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29427
- Priorities of test instabilities (docs about it)
- Test instabilities are prioritized as Critical and become blocker as soon as we notice that they are newly introduced
- Feature freeze extended until Jan 31, 2023
- Switching to weekly calls as we're getting closer to the feature freeze?
- Yes, switch to weekly will happen. Next meeting will be Jan 17 due to Chinese New Year coming up and the feature freeze happening soon
- Votes on FLIPs are stalled due to outstanding votes
- Qingsheng Ren will reach out to Martijn Visser about it
- Share in the Slack dev channel to get more people to look into it
- leader elections lacks test coverage (FLIP-285,
)Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-26522 - No issues popped up on the mailing list since 1.16
2023-01-17
- Build instabilities (all 1.17 test instabilities with a priority >=Major)
- Blockers
- Performance regressions
→ In progressJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30623
→ In progressJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30625
→ In progressJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30624
→ Qingsheng will look into this one after FLINK-29427 (see below)Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29405
→ Qingsheng currently is looking into thisJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29427
- Performance regressions
- Martijn Visser to check open critical test stabilities to determine if some need to be assigned / if we're OK in the overall direction
- FLIP-272: Generalized delegation token support has been merged in and a blog post will be written and published about it, prior to the 1.17 release
- https://github.com/apache/flink/pull/21606 has been merged into Flink, but should also be taken into account for externalized connectors - This is tracked under
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30639
- Blockers
2023-01-24
- Build instabilities (all 1.17 test instabilities with a priority >=Major)
- Performance regressions
→ In progressJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30623
→ In progressJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30625
→ In progressJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30624
- Blockers
→ In progress, needs a status updateJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29405
→ In progress, needs a status updateJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30727
→ ResolvedJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30328
→ In progress, needs a status updateJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29427
→ Martijn Visser is looking into this oneJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30618
→ Martijn Visser is looking into this oneJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30733
- Performance regressions
2023-01-31
- Today is feature freeze day
- 26 features / improvements are in for Flink 1.17 (47 in at Flink 1.16, 20 for Flink 1.15, 27 for Flink 1.14)
- 5 features are still listed as expected to be completed, but are not yet in, 4 of them have been merged and are writing documentation, Martijn Visser to check/update for the status of these items.
- Blockers:
→ Matthias will look into thisJira server ASF JIRA columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,customfield_12311032,customfield_12311037,customfield_12311022,customfield_12311027,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,Priority,Priority,Priority,Priority,priority,status,resolution serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30881
→ Leonard review PASS, waiting CI greenJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29427
→ Fixed by QingshengJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29405
→ Should be resolved, pending validation by the benchmarks (related to FLINK-30624). Benchmarks look to be improved, downgraded to Critical.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30625
→ Should be resolved, pending validation by the benchmarks (related to FLINK-30625). Benchmarks look to be improved, downgraded to Critical.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30624
→ Martijn to reach out to Dong, Rui Fan, Piotr. We're planning to give them until Friday the 3rd of February to come to a conclusion on this ticket; if no consensus is achieved, then the original commit that introduced the regression should be reverted.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30623
→ Matthias to check if this has already been resolved via another ticket (multiple related tickets)Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30826
→ [Critical] PR to update buffers for the test has been merged, will be continued to monitor. Test downgraded to CriticalJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30727
→ [Critical] Downgraded to Critical as it's only a test-related issue and doesn't indicate a bug in production.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30846
→ [Major] Test downgraded to Major, If the test fail again contributor will increase the waiting interval for this test as a solution.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30844
→ [Major] Leonard downgrade the issue priority to Major as it's a known slack plugin issueJira server ASF JIRA columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,customfield_12311032,customfield_12311037,customfield_12311022,customfield_12311027,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,Priority,Priority,Priority,Priority,priority,status,resolution serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30870
- Martijn Visser to communicate to the Flink community this evening that the feature freeze has started and we plan to cut the release branch at the end of this week (Friday 3rd of February).
2023-02-07
master
is stabilized enough to cut therelease-1.17
branch- FLINK-30921 - The Azure apt mirror instabilities seem to have been resolved for now.
- FLINK-30908 - The issue turned out to be a problem that existed in previous releases
- FLINK-30921 - The Azure apt mirror instabilities seem to have been resolved for now.
- Release branch is going to be cut today by Leonard Xu
- Release testing will be announced: 2 weeks will be planned for this
2023-02-14
- Status update on release testing efforts
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30926 - End of cross-team testing date is 21st of February 2023. We will monitor the status throughout the week and hopefully conclude everything next week.
- When a feature is cross-team tested, the icon needs to be changed from to to indicate that the testing has been completed
- We always look for volunteers: picking up a cross-team testing task is much appreciated.
- Proposal: Create Jira issues for release management tasks to document what was done to improve review-ability (alternatively, add expected output to release documentation)
- Do this from now on for the next steps in release management (create release candidate etc.)
- Test instabilities:
: BlockerJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31036
: Starts failing more regularly again. Qingsheng Ren to ping Godfrey He and Yunhong ZhengJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18356 - FLINK-30972 (openssl version update necessary) continues to fail because ofFLINK-30965 (repo-sync doesn't pick up 1.15 changes anymore)
2023-02-21
- Open Blocker issues
- Example Release Jira issues:
>Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31146 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31154 - Release testing not finished, yet
- Decision on RC creation moved to next week's Flink release sync call
- Other issues:
: Issue in 1.15.3 unexplainedJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31133
: 1.17 issue in HiveJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31092
: no updates on the Slack bot instability so farJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30733
: OOM issue - waiting for Gabor's responseJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31134
(Kafka infrastructure umbrella ticket)Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31145
2023-02-28
- Blockers: are these really blockers or do we want to move the release forward?
→ Qingsheng Ren to check the status with Shengkai FangJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31092
→ Qingsheng Ren to check the status with Shengkai FangJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31104
→ Qingsheng Ren to check the status with Shengkai FangJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30978
- Other issues:
: Revisit OOM in Kafka e2e test in 1.15.3 - we're keeping it as critical for now. There are no other artifacts that we can investigate. The test doesn't involve table-runner code (therefore, isn't connected toJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31134
). → Martijn Visser to check internallyJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18356 - Dependabot alerts are affecting all commiters now
- Release managers to review draft of 1.17 announcement
- Release 1.17 preparation:
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31146
2023-03-07
- Ubuntu mirror instabilities:
Matthias Pohl will follow-up on that oneJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30921 - API backwards compatibility:
Do we have additional documentation on API compatibility? Leonard Xu will do another pass over the ticket comments to confirm the findingsJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31167 - Announcement: FLIPs missing: FLIP-272: Generalized delegation token support, FLIP-217: Support watermark alignment of source splits Leonard Xu reaches out to contributors to add something related to the announcement
- Instabilities (blocker query):
:Leonard Xu has pinged luoyuxia to look into this issue.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31351
: Downgraded→ [Critical] PR ready, flink-runtime tests will be made sequential ,Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31278
: Downgraded→ [Critical] Leonard Xu has pinged Godfrey He to look into this issue.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31339
: Downgraded→ [Critical]Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31342
: Downgraded→ [Critical]Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31341
: Downgraded→ [Critical] Matthias PohlJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31354
: Downgraded→ [Critical] Leonard Xu has pinged Dian Fu to look into this issue.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31355
2023-03-14
- Flink RC2 for 1.17 is available, needs checkers
2023-03-21
- RC3 for 1.17.0 was created and is in voting stage (1 binding vote miss)
- Robert Metzger agreed to assist with the license check
2023-03-23
- Flink 1.17.0 is officially released today!
Retrospective:
From Qingsheng Ren
- As discussed in the mailing list, we need to trigger a final patch version for 1.15 after releasing 1.17. Some cleanup steps need to be reviewed and changed, such as removing 1.15 data from svn, CI, flink-docker etc. See
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31570 - I like the idea made by Matthias Pohl that we track TODOs for releasing on JIRA 👍 I used it as a checklist to make sure we don't miss anything. Also it helps collaborating, as we can divide works across RMs easily by assigning JIRA tickets.
From Matthias Pohl
Google Meet might not be the best choice for the release sync. We need to be able to invite attendees even if the creator of the meeting isn't available (maybe try Zoom or even Jitsi as an OpenSource alternative instead?)
- Release sync every 2 weeks and a switch to weekly after feature freeze felt reasonable
Slack worked well as a collaboration tool to document the monitoring tasks (#builds, #flink-dev-benchmarks) in a team with multiple release managers
The Slack Azure Pipeline bot seems to be buggy. It swallows some build failures. It's not a severe issue, though. We created #builds-debug to monitor whether it's happening consistently. The issue is covered in
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30733 We experienced occasional issues in the manual steps of the release creation in the past (e.g. japicmp config was not properly pushed). Creating Jira issues for the release helped to make the release creation more transparent and made the steps more reviewable. Additionally, it helped to distribute subtasks to different people with Jira being the tool for documentation and synchronization. That's especially helpful when there is more than one person in charge of creating the release.
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31146 - RCs
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31154 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31578 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31583
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31562 Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31567
We had backports/merges without PRs happening by committers occasionally during the 1.17 release which broke master/release branches (probably, changes were done locally before merging which were not part of the PR to have a faster backport experience). It might make sense to remind everyone that this should be avoided. Not sure whether we want/can restrict that.
We observed a good response on fixing test instabilities by the end of the release cycle but had some long running issues earlier in the cycle which caused extra efforts on the release managers due to reoccurring test failures.
Release testing picked up “slowly”: Initially, we planned 2 weeks for release testing. But there was not really any progress (tickets being created and worked on) in the first week. In the end, we had to extend the phase by another week resulting in 3 instead of 2 weeks of release testing. I guess we could encourage the community to create release testing tasks earlier and label them properly to be able to monitor the effort. That would even enable us to do release testing for a certain feature after the feature is done and not necessarily only at the end of the release cycle.
- Manual test data generation is tedious (
). But this should be fixed in 1.18 withJira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-31593
being almost done.Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-27518 - We started creating documentation for release management. The goal is to collect what tasks are there to help support a Flink release to encourage newcomers to pick up the task.
From Leonard Xu
- We can keep RC0 (a non-votable one) in future releases, as an initial version for developers to validate, so that some issues could be found earlier and avoid repeatedly canceling and re-creating RCs.
From Martijn Visser
- We should be more careful for commits without a PR / green CI, which brought some problem at the end of 1.17 release cycle. There might not be possible to totally ban this, but we could give an reminder to committers.