Table of contents

Jira Boards

Flink 1.17 Burndown: https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=572

Sync meeting

The sync meeting is happening every second Tuesday starting on the 1st of November 2022 at 9am CEST / 4pm China Standard Time / 8am UTC. 
As we are getting closer to the feature freeze we will do the meeting on a weekly base.

Feel free to join on Google Meet. Local dial-in numbers can be found at https://tel.meet/wcx-fjbt-hhz?pin=1940846765126

Timeline

Feature Freeze
January, 17th of 2023, end of business CEST. 

Chinese new year is 22 January: 1+ week holiday. Ppl will be back in early February

Release
Mid of March 2023

Highlight features

please feel free to add/suggest.

Features

List of features announced by contributors and committers that are likely to be ready for the feature freeze:n

NOTICE: It's preferred if only new features end up there and not all bugs/tasks separately, so that the page is not over bloated. Of course, unless fixing a bug is a really big or important one equivalent to implementing a completely new feature. A good rule of thumb would be that each entry in the page could (but does not have to) be later on included in a release blog post.

Legend

State

symbol	meaning	comment
	validated	through cross team testing
	done	well documented with a complete test coverage
	will make it	there is no reason this effort should not go into 1.17
	in danger	there are some concerns the effort could be ready for the feature freeze of 1.17
	very unlikely	there are severe concerns the effort could make it to 1.17
	won't make it	it was decided against adding this for the 1.17 release. working on the effort has been stopped.
	state unclear
	independent	as the artefact could be released independent of Apache Flink

X-Team verification

symbol	meaning
	done
	not required

Feature Stage

Please align with the list on the Apache Flink Roadmap (https://flink.apache.org/roadmap.html).

• MVP: Have a look, consider whether this can help you in the future.
• Beta: You can benefit from this, but you should carefully evaluate the feature.
• Ready and Evolving: Ready to use in production, but be aware you may need to make some adjustments to your application and setup in the future, when you upgrade Flink.
• Stable: Unrestricted use in production
• Reaching End-of-Life: Stable, still feel free to use, but think about alternatives. Not a good match for new long-lived projects.
• Deprecated: Start looking for alternatives now 

Summary

Numbers are based on the items in the list below, not on the tickets 

								∑	Remaining weeks
2022-11-01									11
2022-11-15									0
2022-11-29	4	10	0	0	0	23	0	37	7
2022-12-13	7	24	1	0	1	9	0	42	5
2022-12-27	7	25	1	0	1	8	0	42	3
2023-01-03									2
2023-01-10									1
2023-01-17									0

List

Feel free to add categories.

Runtime
Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29801	Hang Ruan			FLIP in voting
Shuffle
Jira server ASF JIRA columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29766	Weijie Guo	Xintong Song				100%	10-01-2023
Adaptive Batch Scheduler supports hybrid shuffle mode	Weijie Guo	Xintong Song
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30469	Yuxin Tan	Xintong Song				50%	09-01-2023
Further improvement of production availability of hybrid shuffle	Weijie Guo	Xintong Song				50%	10-01-2023
Deployment & Cluster Coordination
Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified
REST API
Jira server ASF JIRA columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-27060	ConradJam						15-11-2022	15-11-2022	15-11-2022
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30583	Rui Fan	Xintong Song		The first stage is finished, and the second stage is improvement, it will be finished in the next version.		100%	09-01-2023	05-01-2023	09-01-2023
Reactive Mode
Scheduler
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29663	Lijie Wang	Zhu Zhu				80%	03-01-2022
FLIP-281: Speculative execution for sink	Biao Liu	Zhu Zhu		FLIP in discussion		30%	03-01-2022
FLIP-283: Use adaptive batch scheduler as default scheduler for batch jobs	Junrui Lee	Zhu Zhu		FLIP in discussion		30%	03-01-2023
Misc
OLAP
State backend
Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified
FLIP-263: Improve resolving schema compatibility Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29844	Hangxiang Yu	Yuan Mei				80%	10-1-2023
Improve File Management in State Backend	Zakelly Lan  Yanfei Lei  Han Yin	Yuan Mei				5%	13-12-2022
Improve the serializer performace of state change of changelog Jira server ASF JIRA columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30345	Hangxiang Yu	Roman Khachatryan Yuan Mei				100%	10-1-2023
Allow to configure Changelog Storage per program Jira server ASF JIRA columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-26372	Roman Khachatryan Yanfei Lei   Hangxiang Yu	Yuan Mei				70%	10-1-2023
Add a metric for back-pressure from the ChangelogStateBackend Jira server ASF JIRA columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-24402	Yanfei Lei	Yuan Mei				40%	13-12-2022
Frocksdb cannot run on Apple M1 Jira server ASF JIRA columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-24932	Yun Tang  Yanfei Lei	Yuan Mei  Yun Tang				90%	13-12-2022
Checkpoint
Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified
Benchmark Generic Incremental CP + UC + BD	Rui xia Yanfei Lei  Hangxiang Yu	Yuan Mei
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-26803	Rui Fan	Piotr Nowojski				70%	10-12-2022
Benchmark
Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified
Improve benchmark stability Jira server ASF JIRA columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29825	Yanfei Lei	Yuan Mei				50%	3-1-2023
API
Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29740	Martijn Visser		Deprecated			100%	08-11-2022	08-11-2022	08-11-2022
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-28641	Martijn Visser		Removed			40%	08-11-2022
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-24456
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18647	Yun Gao		FLIP Discussion	Need to be postponed to the next release			10-01-2023
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29668	Martijn Visser		Removed			100%	14-11-2022	14-11-2022	14-11-2022
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30050
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-25509	Hang Ruan	Dong Lin	Coding
SQL
Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified
General
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29942	Godfrey He ,@Yunhong Zheng					70%	2023-01-10
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-28016	Chesnay Schepler
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29281	Chesnay Schepler
Calcite Update
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-20873	Sergey Nuyanzin
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-21239	Sergey Nuyanzin
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29932	Sergey Nuyanzin
Flink Dialect
ALTER TABLE API Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-21634	Shengkai Fang					90%	2023-01-10
DELETE/UPDATE API (FLIP-282)	luoyuxia			voting			2023-01-10
Hive Dialect
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29635	luoyuxia					60%
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29717	ron liu					80%	2023-01-10
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-26603	luoyuxia
SQL Gateway
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29941	Shengkai Fang  , Zelin Yu					70%	2023-01-10
QE
FLIP-280: Introduce a new explain mode to provide SQL advice	Jane Chan			voting			2023-01-10
QO
FLINK-27591 - Improve the plan for batch queries when statistics is unavailable OPEN	Godfrey He ,@Yunhong Zheng					40%
API/Python
Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29155	HuangXingbo					100%	10-01-2023
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29833	HuangXingbo					0%	10-01-2023
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-28957	HuangXingbo					70%	10-01-2023
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29421	HuangXingbo					0%	10-01-2023
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-21223	HuangXingbo					100%	10-01-2023
Machine Learning
Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified
CEP
Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified
Web
Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified
Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29995	Yu Chen	Yun Tang

Status / Follow-ups

2022-11-01

Agenda

• Kickoff
• Keeping the state of features updated (ideally before the sync)
• Blockers

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-24119

• Build stability
• Are there any (new) contributors who need a PR reviewed or merged? And if yes, who can help out?

2022-11-15

Agenda

• Kickoff
• Keeping the state of features updated (ideally before the sync)
• Blockers

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29387

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29315

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29818

• Build stability
  • Number of growing test stability issues with "Exit code 137 errors"

    • Jira
      server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18356

    • Jira
      server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29899

    • Jira
      server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-26674

• Are there any (new) contributors who need a PR reviewed or merged? And if yes, who can help out?

2022-11-29

• Build instabilities

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-28766

    : Anton has some new findings on that issue and will get back on it
  • Pulsar-related issues:

    • Jira
      server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29835

    • Jira
      server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29755

    • Jira
      server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29836

  • Python-related issues:

    • Jira
      server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29461

      Matthias Pohl pings Xingbo Huang / Dian Fu 

    • Jira
      server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-26974

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18356

    : OOM errors are most likely being caused by flink-table-planner (Godfrey He might be a person to reach out to about it)

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29427

    : Qingsheng Ren will look into the PR

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-27916

    Matthias Pohl will ping contributors on the issue - but it's not that urgent since it's not failing that frequently 
• Other topics:
  • Externalizing Pulsar connector (test instabilities): Martijn Visser  is working on externalizing the code base
  • Performance test monitoring: Discussion is happening on the mailing list
  • Externalizing connectors in general is work-in-progress but looks good
  • Public CI documentation can be improved
    • Matthias Pohl will work on a first approval

2022-12-13

• Build instabilities

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29405

    → Qingsheng to have a look at the PR
  • Pulsar connector has been synced to external connector repository. Martijn Visser to open a PR to remove connector from master

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18356

    → Qingsheng to ping Godfrey

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18356

    → The PR is still failing for the same issue as this PR should fix. We should ping the author to have a look first

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-27916

    → Martijn to ping Thomas once more

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-26974

    → Xingbo is working on this
• How to have monitoring and quality control for the externalized connectors → Need to have a discussion on the Dev mailing list. Martijn Visser to make a proposal and open a discussion thread on this topic.

2022-12-27

• Meeting skipped due to Christmas holiday/sick leaves
• Discussion started on moving the feature freeze from Jan 17 to Jan 31 due to pandemic situation in China (see dev ML discussion thread)
• Pulsar connector has been externalized. Pulsar-related test instabilities were disabled in release-1.16  and release-1.15  (see

  Jira
  server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-30351

  and parent task)

2023-01-10

• Build instabilities (all 1.17 test instabilities with a priority >=Major)

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-18356

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-26974

  • Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key FLINK-29427

• Proposal: Priorities of test instabilities (docs about it)
  • Blocker: New test instabilities which were introduced in unreleased versions only
  • Critical: New test instabilities that also affect released versions; Test instabilities that pop up more than two times per month (because they affect CI maintenance)
  • Major: Test instabilities that popped up after the version was released but happen rarely
• Feature freeze extended until Jan 31, 2023
• Switching to weekly calls as we're getting closer to the feature freeze?