THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
Table of contents
Jira Boards
Flink 1.19 Burndown: 1.19 Burndown - Agile Board - ASF JIRA (apache.org)
Sync meeting
- The release sync will start from Nov 14th, 2023, at 9am (CET, UTC+1) and 4pm (UTC+8). The release sync happens bi-weekly at first, and will be adjusted to weekly as we approaching the feature freeze date.
- Feel free to join on Google Meet, and attendees are encouraged to fill out the topics to be discussed at the bottom of this page a day in advance to make it easier for everyone to understand the background of the topic.
Timeline
- Feature Freeze
- Jan 26, 2024, 00:00 CET(UTC+1)
- Release
- End of Feb 2024
Highlight features
please feel free to add/suggest.
Features
List of features announced by contributors and committers that are likely to be ready for the feature freeze:n
NOTICE: It's preferred if only new features end up there and not all bugs/tasks separately, so that the page is not over bloated. Of course, unless fixing a bug is a really big or important one equivalent to implementing a completely new feature. A good rule of thumb would be that each entry in the page could (but does not have to) be later on included in a release blog post.
Legend
State
| symbol | meaning | comment |
|---|---|---|
 | validated | through cross team testing |
 | done | well documented with a complete test coverage |
 | will make it | there is no reason this effort should not go into 1.19 |
 | in danger | there are some concerns the effort could be ready for the feature freeze of 1.19 |
 | very unlikely | there are severe concerns the effort could make it to 1.19 |
 | won't make it | it was decided against adding this for the 1.19 release. working on the effort has been stopped. |
 | state unclear | |
 | independent | as the artifact could be released independent of Apache Flink |
X-Team verification
| symbol | meaning |
|---|---|
| done |
| not required |
Feature Stage
Please align with the list on the Apache Flink Roadmap (https://flink.apache.org/roadmap.html).
- MVP: Have a look, consider whether this can help you in the future.
- Beta: You can benefit from this, but you should carefully evaluate the feature.
- Ready and Evolving: Ready to use in production, but be aware you may need to make some adjustments to your application and setup in the future, when you upgrade Flink.
- Stable: Unrestricted use in production
- Reaching End-of-Life: Stable, still feel free to use, but think about alternatives. Not a good match for new long-lived projects.
- Deprecated: Start looking for alternatives now
Summary
Numbers are based on the items in the list below, not on the tickets
| | | | | | | | ∑ | Remaining weeks | |
| 2023-11-28 | 2 | 11 | 21 | 9 | ||||||
| 2023-12-12 | 3 | 11 | 23 | 7 | ||||||
| 2023-12-26 | 4 | 15 | 27 | 5 | ||||||
| 2024-01-09 | 4 | 15 | 1 | 27 | 3 | |||||
| 2024-01-23 | 15 | 7 | 2 | 1 | 4 | 29 | 1 | |||
| 2024-01-30 | 24 | 0 | 2 | 1 | 4 | 31 | 0 | |||
| 2024-02-06 |
List
Feel free to add categories.
| Runtime | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | Volunteers for testing |
| Shuffle | |||||||||||
| Coordination | |||||||||||
| FLIP-379: Dynamic source parallelism inference for batch jobs | | 2024-01-23 | |||||||||
| Accepted | | 2024-01-22 | ||||||||
| Done | All features are merge, it only left a little test and doc, so marking it done first. | | 2024-01-22 | |||||||
| FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces | Done | | 2024-01-18 | | |||||||
FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter | Piotr Nowojski | Done | | 2024-01-24 | | ||||||
| Done | | 2024-01-24 | | ||||||||
| FLIP-383: Support Job Recovery for Batch Jobs | Accepted | | 2024-01-22 | ||||||||
| State & Checkpoint | |||||||||||
| | 2024-1-15 | |||||||||
| | 2024-1-24 | |||||||||
REST | |||||||||||
| Configuration | |||||||||||
| 2024-01-26 | ||||||||||
FLINK-33712 - FLIP-391: Deprecate RuntimeContext#getExecutionConfig | | 2024-01-16 | |||||||||
FLINK-33581 - FLIP-381: Deprecate configuration getters/setters that return/set complex Java objects | | 2024-01-16 | |||||||||
| Done | | 2024-01-22 | ||||||||
| SQL | |||||||||||
| Name/JIRA Issue | Responsible Contributor | Reviewer / committer available | Feature Stage | Note | State | % | Updated | Implemented | Documented | X-team verified | |
| | 2024-01-30 | |||||||||
| @Yangze Guo | | |||||||||
| FLIP-376: Add DISTRIBUTED BY clause | | 2024-01-30 | |||||||||
| no new flip needed, continue the work on FLINK-24024 & FLINK-20281 | | 2024-01-26 | FLINK-20281: N/A | ||||||||
| FLIP-393: Make QueryOperations SQL serializable | Done | | 2024-01-18 | | N/A | ||||||
| FLIP-387: Support named parameters for functions and call procedures | @Feng Jin | xuyang | need documentation | | 2024-01-30 | ||||||
| Under discussion | | 2024-01-18 | ||||||||
| need documentation | | 2024-01-30 | | ||||||||
| Timo Walther | Needs documentation | | 2024-02-05 | | |||||||
| Connectors | |||||||||||
| | 2024-01-22 | |||||||||
| Pulsar connector has formal release already | | 2024-01-23 | ||||||||
| only left for changes in kafka connector | | 2024-01-22 | ||||||||
FLIP-371: Provide initialization context for Committer creation in TwoPhaseCommittingSink | Done | | 2024-01-18 | ||||||||
| Mark it done since main part of the new api been merged (remaining tests and doc) | | 2024-01-23 | ||||||||
| | ||||||||||
| Resource Management | |||||||||||
Runtime | |||||||||||
| Rui Fan | RocMarshal | Done | | 2023-12-04 | ||||||
A series of optimizations to reduce the memory and cpu usage for large StreamOperator | Rui Fan | | 2023-11-23 | ||||||||
Misc | |||||||||||
| DONE, only docs left | | 2024-01-23 | ||||||||
(will create a FLIP) | | 2024-01-23 | |||||||||
Status / Follow-ups
11/14/2023
- Is Google Meet OK for everyone? - ok
- The release sync meeting time, currently Europe has gone into winter time(CET, UTC+1), is 9am ok? - ok
- Daily work divisions
- JIRA ticket (Mainly focus on new blockers, progress of existing blockers each week) lincoln lee
- CI (Failures reported in #build channel and not reported) Martijn Visser
- Benchmark (Check if there's regression in the #flink-dev-benchmarks channel) Yun Tang
- Mailing list monitoring Jing Ge
- This division doesn't mean that other RMs are now allowed to do the work. All RMs should take responsibilities on all of these daily issues if they are available.
- Feature freezing date. - Jan 26, 2024, 00:00 CET(UTC+1)
- Retrospective of 1.18
11/28/2023
- Blockers
- discuss thread - back to ml discussion
- documentation build failures - closed
- GitHub Actions Migration discussion in dev ML (FLIP-396)
- flink-shaded driven by Sergey Nuyanzin
- As suggested by Xintong Song, we encourage attendees to fill out the topics a day in advance to make it easier for everyone to understand the background of the topic.
12/12/2023
- Current status of 1.19 Features lincoln lee - updated
- Existing blocker issues lincoln lee
- discuss thread - closed
- (potential blocker)
12/26/2023
- Current status of 1.19 Features lincoln lee - updated
- 5 weeks remaining, total 27, done 4, will make it 15
- Existing blocker issues lincoln lee
- has been fixed
- Skip this sync meeting? - no response until 12/25, skipped finally
01/09/2024
- Current status of 1.19 Features lincoln lee - updated
- 3 weeks remaining, total 27, done 4, will make it 15, won't make it 1
- Existing blocker issues lincoln lee
- sergy reported a breaking change, will be fixed soon Wencong Liu
01/23/2024
- Current status of 1.19 Features lincoln lee
- Jan 26th is the planned feature freeze day, total 28, done 15, will make it 6, won't make it 4;
- Existing blocker issues lincoln lee
- Sergey Nuyanzin Zakelly Lan
- Matthias Pohl
-
the disk-full issue was addressed, should be stable for now
- Weird build on Friday (Jan 19): https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=56601&view=results (probably caused by infrastructure issues; but AFAIU, they were resolved the day before?)
- Whether to delay feature freeze day？ Yun Tang
- Build pipeline stability
- : python stage reaches 4h timeout
-
:
AsyncSinkWriterThrottlingTestquite unstable recently - : macos wheels build seems to be unstable (multiple open issues)
- / : new feature
01/30/2024
- Current status of 1.19 Features lincoln lee
- We extend the feature freeze day to Jan 30th, total 31, done 24;
- FLIP-376: Add DISTRIBUTED BY clause could be done this week: do we want to wait, or do we want to cut the release branch? Martijn Visser → Will be postponed to 1.20
- Existing blocker issues lincoln lee
- Sergey Nuyanzin Zakelly Lan
- Matthias Pohl
- & @yunfengzhou
- When to cut release-1.19 branch?
- Cross-team release testing work
02/06/2024
- Blockers:
- &
- pr reviewing jiabao.sun
- not yet clarified whether it's a production code or test issue
- Cut release-1.19 branch
- fyi: GitHub Actions e2e tests seem to have disk space issues causing multiple test failure (not present in Azure)
- Cross-team release testing
- total 26 flip/features, 15 confirmed, 7 not required, 4 wait for response(authors: Dawid Wysakowicz, Peter Vary, Alan Sheinberg)
- testing instructions ready: 11 (1 completed, 3 assigned, 7 unassigned), wait for doc ready: 2
- more completed flips found, will update the testing ticket (done, 6 Instructions tickets added)
02/13/2024
- Not verified Jira issues:
- CI instability issues:
- Nightly build for 1.19 doesn't seem to be picked
- Cross-team release testing
- wait for response: there's some authentication problem in current kanban dashboard, use filter on Jira search 'project = FLINK AND type = Sub-task AND parent = FLINK-34285 AND labels is EMPTY AND status = OPEN ORDER BY updatedDate'
- wait for doc ready:
- testing tickets: Jira filter condition 'project = FLINK AND type = Sub-task AND parent = FLINK-34285 AND labels = release-testing ORDER BY updatedDate'
02/20/2024
- Not verified Jira issues:
- CI instability issues:
- Cross-team release testing
wait for response: there's some authentication problem in current kanban dashboard, use filter on Jira search 'project = FLINK AND type = Sub-task AND parent = FLINK-34285 AND labels is EMPTY AND status = OPEN ORDER BY updatedDate'- wait for doc ready:
- testing tickets: Jira filter condition 'project = FLINK AND type = Sub-task AND parent = FLINK-34285 AND labels = release-testing ORDER BY updatedDate'
- 15 total, 8 closed, 4 unassigned, 3 testing
- When to create rc? –- after release testing finished and no blockers