Feel free to join on Google Meet. Local dial-in numbers can be found at https://tel.meet/wcx-fjbt-hhz?pin=1940846765126

Timeline

Feature Freeze
- ~~January, 17th of 2023, end of business CEST.~~

...

- January 31st, 2023, end of business CEST (ML discussion on change)
Release
- ~~Mid of March 2023~~
- End of March 2023

Chinese new year is 22 January: 1+ week holiday. Ppl will be back in early FebruaryRelease
Mid of March 2023

Highlight features

please feel free to add/suggest.

...

symbol	meaning	comment
	validated	through cross team testing
	done	well documented with a complete test coverage
	will make it	there is no reason this effort should not go into 1.17
	in danger	there are some concerns the effort could be ready for the feature freeze of 1.17
	very unlikely	there are severe concerns the effort could make it to 1.17
	won't make it	it was decided against adding this for the 1.17 release. working on the effort has been stopped.
	state unclear
	independent	as the artefact artifact could be released independent of Apache Flink

...

Numbers are based on the items in the list below, not on the tickets

1107522023-01-17

									∑	Remaining weeks
2022-11-01										13
2022-11-15										11
2022-11-29		4	10	0	0	0	23	0	37	9
2022-12-13		7	24	1	0	1	9	0	42	7
2022-12-27	3	2023-01-03			7	25	1	0	1	8	0	42	5
2023-01-10	1		9	27	0

List

Feel free to add categories.

0	3	5	0	44	3
2023-01-17		13	23	0	0	3	3	2	44	2
2023-01-24		16	21	0	0	3	3	2	45	1
2023-01-31		26	5	0	0	16	0	0	47	0
2023-02-14	5	23	3

List

Feel free to add categories.

RuntimeName/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verifiedShuffle

Jira

server	ASF JIRA
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-29766

Weijie Guo

Xintong Song

75%27-12-2022Adaptive Batch Scheduler supports hybrid shuffle modeWeijie Guo Xintong Song

FLIP-266: Simplify network memory configurations for TaskManagerYuxin Tan Xintong Song PoC done, FLIP under discussion

20%27-12-2022Further improvement of production availability of hybrid shuffle

Weijie Guo

Xintong Song

20%27-12-2022Deployment & Cluster CoordinationName/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verifiedREST API

Runtime

Name/JIRA Issue

Responsible Contributor

Reviewer / committer available

Feature Stage

Note

State

%

Updated

Implemented

Documented

X-team verified

Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-29801

Hang Ruan

FLIP in voting

Shuffle

Jira

server	ASF JIRA
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

27060

29766

ConradJam

Weijie Guo

15-11-202215-11-2022

Xintong Song

100%

10-01-2023

14-02-2023

15-11-2022

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

30185

30938

AdaptiveBatchScheduler should supports early consumption for dynamic graph.

Weijie Guo

Rui FanReactive ModeScheduler

Xintong Song

60%10-12-2022

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

29663

30469

80%

100%

03

17-01-

2022

2023

FLIP-281: Speculative execution for sinkBiao Liu

Zhu Zhu

FLIP in discussion

30%03-01-2022FLIP-283: Use adaptive batch scheduler as default scheduler for batch jobs@Junrui Li

Zhu Zhu

FLIP in discussion

30%

17-01-2023	10-02-2023	self-test
Further improvement of production availability of hybrid shuffle	Weijie Guo	Xintong Song				100%	17

03

-01-2023

MiscOLAPState backend

Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified
17-01-2023		self-test
Deployment & Cluster Coordination

FLIP-263: Improve resolving schema compatibility

REST API

Jira

server	ASF JIRA
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

29844

27060

Hangxiang Yu

ConradJam

Yuan Mei

20%13-12-2022

Improve File Management in State Backend

Zakelly Lan

Yanfei Lei

Han Yin

Yuan Mei

5%13-12-2022

Improve the serializer performace of state change of changelog

JiraserverASF JIRAcolumnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolutioncolumnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution

100%

20-02-2023

15-11-2022

self-test

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-30583

Rui Fan

Xintong Song

The first stage is finished, and the second stage is improvement, it will be finished in the next version.

100%

09-01-2023

05-01-2023

09-01-2023

Reactive Mode

n/a

Scheduler

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

30345

29663

Hangxiang Yu

Lijie Wang

Roman Khachatryan Yuan Mei

Zhu Zhu

80%

100%

13

17-

12

01-2022

Allow to configure Changelog Storage per program

Jira

server	ASF JIRA

columnIdscolumnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution

issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution

serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

26372

50%13-12-2022

Add a metric for back-pressure from the ChangelogStateBackend

30725

Biao Liu

Zhu Zhu

100%

31-01-2022

Jira

server	ASF JIRA

columnIdscolumnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution

issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution

serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

24402

Frocksdb cannot run on Apple M1

Jira

31005

Yanfei Lei

Yuan Mei

40%13-12-2022

Jira

server	ASF JIRA

columnIdscolumnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution

issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution

serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

24932

30682

State backend
100%	31-01-2023
Misc
n/a
OLAP
n/a

90%13-12-2022Checkpoint


Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified

Benchmark Generic Incremental CP + UC + BD

Rui xia

Yanfei Lei

Hangxiang Yu

Yuan Mei

FLIP-263: Improve resolving schema compatibility

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

26803

29844

70%

80%

10

31-

12

1-

2022

2023

Improve File Management in State Backend

Zakelly Lan

Yanfei Lei

Han Yin

Yuan Mei

5%

31-1-2023

Improve the serializer performace of state change of changelog

BenchmarkName/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verifiedImprove benchmark stability

Jira

server	ASF JIRA
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

29825

30345

Yanfei Lei

Hangxiang Yu

Roman Khachatryan Yuan Mei

30%

100%

13

31-

12

1-

2022

2023

APIName/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified

self-test

Allow to configure Changelog Storage per program

Jira

server	ASF JIRA

serverId5aa69414-a9e9-3523-82ec-879b028fb15bkeyFLINK-29740

Martijn Visser

Deprecated

100%08-11-202208-11-202208-11-2022 JiraserverASF JIRA


columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution

serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

28641

Martijn Visser

Removed

40%08-11-2022

26372

70%

31-1-2023

Add a metric for back-pressure from the ChangelogStateBackend

Jira

server	ASF JIRA
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

24456

24402

Yanfei Lei

Yuan Mei

40%

31-1-2023

Frocksdb cannot run on Apple M1

Jira

server	ASF JIRA
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	5aa69414-

JiraserverASF JIRAserverId5aa69414-

a9e9-3523-82ec-879b028fb15b
key	FLINK-

18647

24932

Yun

Gao

Tang

FLIP Discussion

13-12-2022

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-29668

Martijn Visser Removed

100%14-11-202214-11-202214-11-2022

Yanfei Lei

Yuan Mei

Yun Tang

100%

30-1-2023

Tested by Martijn Visser

Release FRocksDB 6.20.3-ververica-2.0

Jira

server	ASF JIRA

serverId5aa69414-a9e9-3523

columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK

-30050

-30836

Yanfei Lei

Yuan Mei

Yun Tang

100%

31-1-2023

Tested by Martijn Visser

Checkpoint

SQL


Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified

General

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-29942

Godfrey He ,@Yunhong Zheng

35%

Benchmark Generic Incremental CP + UC + BD

Rui xia

Yanfei Lei

Hangxiang Yu

Yuan Mei

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

28016

Chesnay Schepler

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-29281

Chesnay Schepler

Calcite Update

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-20873

Sergey Nuyanzin

26803

Rui Fan

Piotr Nowojski

100%

29 Jan 2023

Benchmark

Name/JIRA Issue

Responsible Contributor

Reviewer / committer available

Feature Stage

Note

State

%

Updated

Implemented

Documented

X-team verified

Improve benchmark stability

JiraserverASF JIRA

Jira

server	ASF JIRA

serverId5aa69414-a9e9-3523-82ec-879b028fb15bkeyFLINK-21239

Sergey Nuyanzin

columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution

serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

29932

29825

Sergey Nuyanzin

Yanfei Lei

Flink Dialect

ALTER TABLE API

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-21634

Shengkai Fang

90%2023-01-03DELETE/UPDATE API (FLIP-282)

luoyuxia

PoC done, FLIP under discussion

2023-01-03

API
Yuan Mei				50%	31-1-2023
Name/JIRA Issue	Responsible Contributor	Reviewer / committer available	Feature Stage	Note	State	%	Updated	Implemented	Documented	X-team verified

Hive Dialect

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

29635

29740

luoyuxia

Martijn Visser

Deprecated

60%server

Jira

100%

08-11-2022

Self-tested

Jira

server

ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

29717

28641

ron liu

Martijn Visser

Removed

40%

31-01-2023

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

26603

24456

luoyuxia

Dawid Wysakowicz

90%

31-01-2022

SQL Gateway

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

29941

18647

Shengkai Fang , Zelin Yu

30%2023

Yun Gao

FLIP Discussion

Need to be postponed to the next release

10-01-

03

2023

QEFLIP-280: Introduce a new explain mode to provide SQL advice

Jane Chan

PoC done, FLIP under discussion

2023-01-03QO

Image RemovedFLINK-27591 - Improve the plan for batch queries when statistics is unavailable OPEN

Godfrey He ,@Yunhong Zheng

40%

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-29668

Martijn Visser

Removed

100%

14-11-2022

Self-tested

API/PythonName/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

29155

30050

HuangXingbo

90%

100%12

31-

12

01-2022

Self-tested

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

29833

25509

HuangXingbo

Hang Ruan

Jira

Dong Lin

0%27-12-2022

Coding

SQL

Name/JIRA Issue

Responsible Contributor

Reviewer / committer available

Feature Stage

Note

State

%

Updated

Implemented

Documented

X-team verified

General

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

28957HuangXingbo

70%27-12-2022

29942

Godfrey He ,@Yunhong Zheng

100%

2023-01-31

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

29421

28016

HuangXingbo

Chesnay Schepler

0%27

31-

12

01-

2022

2023

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

21223

29281

HuangXingbo

Chesnay Schepler

100%12

31-

12

01-

2022

2023

Machine LearningName/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verifiedCEPName/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verifiedWebName/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified

Table API

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-30025

Jing Ge

2023-01-20

Calcite Update

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-20873

Sergey Nuyanzin

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-21239

Sergey Nuyanzin

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

29995

Yu Chen

29932

Sergey Nuyanzin

Yun Tang

Status / Follow-ups

2022-11-01

Agenda

...

	31-01-2023
Flink Dialect
ALTER TABLE API

...

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

...

2022-11-15

Agenda

...

21634

Shengkai Fang

100%

2023-02-14

self-tested

Jira

server

ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

...

30648

...

luoyuxia

...

100%	2023-02-14			self-tested
Hive Dialect

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

...

29635

...

luoyuxia

...

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

...

30951

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

...

29717

...

ron liu

...

100%

2023-01-31

Jira

server	ASF

...

JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

...

2022-11-29

...

26603

luoyuxia

31-01-2022

SQL Gateway

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

...

29941

Shengkai Fang , Zelin Yu

writing doc

100%

2023-01-10

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

...

30936

QE

Jira

server	ASF JIRA
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK

...

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-29461

...

-30650

Jane Chan

100%

2023-01-10

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

...

31025

QO

Image AddedFLINK-27591 - Improve the plan for batch queries when statistics is unavailable OPEN

Godfrey He ,@Yunhong Zheng

Need to be postponed to the next release

40%

2023-1-31

API/Python

Name/JIRA Issue

Responsible Contributor

Reviewer / committer available

Feature Stage

Note

State

%

Updated

Implemented

Documented

X-team verified

Jira

server

ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK

...

-29155

HuangXingbo

100%

10-01-2023

self-test

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

...

29833

HuangXingbo

0%

10-01-2023

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

...

28957

HuangXingbo

100%

19-01-2023

self-test

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK

...

Externalizing Pulsar connector (test instabilities): Martijn Visser is working on externalizing the code base
Performance test monitoring: Discussion is happening on the mailing list
Externalizing connectors in general is work-in-progress but looks good
Public CI documentation can be improved
- Matthias Pohl will work on a first approval

2022-12-13

...

-29421

HuangXingbo

100%

19-01-2023

self-test

Jira

server

ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-

...

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-18356

...

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-18356

...

21223

HuangXingbo

100%

10-01-2023

self-test

Machine Learning

Name/JIRA Issue

Responsible Contributor

Reviewer / committer available

Feature Stage

Note

State

%

Updated

Implemented

Documented

X-team verified

n/a

CEP

Name/JIRA Issue

Responsible Contributor

Reviewer / committer available

Feature Stage

Note

State

%

Updated

Implemented

Documented

X-team verified

n/a

Web

Name/JIRA Issue

Responsible Contributor

Reviewer / committer available

Feature Stage

Note

State

%

Updated

Implemented

Documented

X-team verified

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-27916

...

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-29995

Yu Chen

Yun Tang

100%

Status / Follow-ups

2022-11-01

Agenda

Kickoff
Keeping the state of features updated (ideally before the sync)
Blockers
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-24119
Build stability
Are there any (new) contributors who need a PR reviewed or merged? And if yes, who can help out?

2022-11-15

Agenda

Kickoff
Keeping the state of features updated (ideally before the sync)
Blockers
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-29387
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-29315
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-29818
Build stability
- Number of growing test stability issues with "Exit code 137 errors"
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-18356
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-29899
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-26674
Are there any (new) contributors who need a PR reviewed or merged? And if yes, who can help out?

2022-11-29

Build instabilities
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-28766
  : Anton has some new findings on that issue and will get back on it
- Pulsar-related issues:
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-29835
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-29755
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-29836
- Python-related issues:
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-29461
    Matthias Pohl pings Xingbo Huang / Dian Fu
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-26974
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-18356
  : OOM errors are most likely being caused by flink-table-planner (Godfrey He might be a person to reach out to about it)
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-29427
  : Qingsheng Ren will look into the PR
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-27916
  Matthias Pohl will ping contributors on the issue - but it's not that urgent since it's not failing that frequently
Other topics:
- Externalizing Pulsar connector (test instabilities): Martijn Visser is working on externalizing the code base
- Performance test monitoring: Discussion is happening on the mailing list
- Externalizing connectors in general is work-in-progress but looks good
- Public CI documentation can be improved
  - Matthias Pohl will work on a first approval

2022-12-13

Build instabilities
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-29405
  → Qingsheng to have a look at the PR
- Pulsar connector has been synced to external connector repository. Martijn Visser to open a PR to remove connector from master
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-18356
  → Qingsheng to ping Godfrey
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-18356
  → The PR is still failing for the same issue as this PR should fix. We should ping the author to have a look first
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-27916
  → Martijn to ping Thomas once more
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-26974
  → Xingbo is working on this
How to have monitoring and quality control for the externalized connectors → Need to have a discussion on the Dev mailing list. Martijn Visser to make a proposal and open a discussion thread on this topic.

2022-12-27

Meeting skipped due to Christmas holiday/sick leaves
Discussion started on moving the feature freeze from Jan 17 to Jan 31 due to pandemic situation in China (see dev ML discussion thread)
Pulsar connector has been externalized. Pulsar-related test instabilities were disabled in release-1.16 and release-1.15 (see
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-30351
and parent task)

2023-01-10

Build instabilities (all 1.17 test instabilities with a priority >=Major)
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-18356
  - Godfrey He and Yunhong Zheng are working on it
  - run tests after each other/not in parallel
  - don't reuse JVMs
  - JUnit has feature to log memory consumption
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-26974
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-29427
Priorities of test instabilities (docs about it)
- Test instabilities are prioritized as Critical and become blocker as soon as we notice that they are newly introduced
Feature freeze extended until Jan 31, 2023
Switching to weekly calls as we're getting closer to the feature freeze?
- Yes, switch to weekly will happen. Next meeting will be Jan 17 due to Chinese New Year coming up and the feature freeze happening soon
Votes on FLIPs are stalled due to outstanding votes
- Qingsheng Ren will reach out to Martijn Visser about it
- Share in the Slack dev channel to get more people to look into it
leader elections lacks test coverage (FLIP-285,
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-26522
)
- No issues popped up on the mailing list since 1.16

2023-01-17

Build instabilities (all 1.17 test instabilities with a priority >=Major)
- Blockers
  - Performance regressions
    - Jira
      server ASF JIRA
      serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
      key FLINK-30623
      → In progress
    - Jira
      server ASF JIRA
      serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
      key FLINK-30625
      → In progress
    - Jira
      server ASF JIRA
      serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
      key FLINK-30624
      → In progress
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-29405
    → Qingsheng will look into this one after FLINK-29427 (see below)
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-29427
    → Qingsheng currently is looking into this
- Martijn Visser to check open critical test stabilities to determine if some need to be assigned / if we're OK in the overall direction
- FLIP-272: Generalized delegation token support has been merged in and a blog post will be written and published about it, prior to the 1.17 release
- https://github.com/apache/flink/pull/21606 has been merged into Flink, but should also be taken into account for externalized connectors - This is tracked under
  Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-30639

2023-01-24

Build instabilities (all 1.17 test instabilities with a priority >=Major)
- Performance regressions
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-30623
    → In progress
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-30625
    → In progress
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-30624
    → In progress
- Blockers
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-29405
    → In progress, needs a status update
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-30727
    → In progress, needs a status update
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-30328
    → Resolved
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-29427
    → In progress, needs a status update
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-30618
  → Martijn Visser is looking into this one
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-30733
  → Martijn Visser is looking into this one

2023-01-31

Today is feature freeze day
26 features / improvements are in for Flink 1.17 (47 in at Flink 1.16, 20 for Flink 1.15, 27 for Flink 1.14)
- 5 features are still listed as expected to be completed, but are not yet in, 4 of them have been merged and are writing documentation, Martijn Visser to check/update for the status of these items.

Blockers:

Jira

server	ASF JIRA
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,customfield_12311032,customfield_12311037,customfield_12311022,customfield_12311027,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,Priority,Priority,Priority,Priority,priority,status,resolution
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-30881

→ Matthias will look into this

Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-29427
→ Leonard review PASS, waiting CI green
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-29405
→ Fixed by Qingsheng
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-30625
→ Should be resolved, pending validation by the benchmarks (related to FLINK-30624). Benchmarks look to be improved, downgraded to Critical.
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-30624
→ Should be resolved, pending validation by the benchmarks (related to FLINK-30625). Benchmarks look to be improved, downgraded to Critical.
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-30623
→ Martijn to reach out to Dong, Rui Fan, Piotr. We're planning to give them until Friday the 3rd of February to come to a conclusion on this ticket; if no consensus is achieved, then the original commit that introduced the regression should be reverted.
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-30826
→ Matthias to check if this has already been resolved via another ticket (multiple related tickets)
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-30727
→ [Critical] PR to update buffers for the test has been merged, will be continued to monitor. Test downgraded to Critical
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-30846
→ [Critical] Downgraded to Critical as it's only a test-related issue and doesn't indicate a bug in production.
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-30844
→ [Major] Test downgraded to Major, If the test fail again contributor will increase the waiting interval for this test as a solution.

Jira

server	ASF JIRA
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,customfield_12311032,customfield_12311037,customfield_12311022,customfield_12311027,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,Priority,Priority,Priority,Priority,priority,status,resolution
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	FLINK-30870

→ [Major] Leonard downgrade the issue priority to Major as it's a known slack plugin issue

Martijn Visser to communicate to the Flink community this evening that the feature freeze has started and we plan to cut the release branch at the end of this week (Friday 3rd of February).

2023-02-07

master is stabilized enough to cut the release-1.17 branch
- FLINK-30921 - The Azure apt mirror instabilities seem to have been resolved for now.
- FLINK-30908 - The issue turned out to be a problem that existed in previous releases
Release branch is going to be cut today by Leonard Xu
Release testing will be announced: 2 weeks will be planned for this

2023-02-14

Status update on release testing efforts
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-30926
- End of cross-team testing date is 21st of February 2023. We will monitor the status throughout the week and hopefully conclude everything next week.
- When a feature is cross-team tested, the icon needs to be changed from to to indicate that the testing has been completed
- We always look for volunteers: picking up a cross-team testing task is much appreciated.
Proposal: Create Jira issues for release management tasks to document what was done to improve review-ability (alternatively, add expected output to release documentation)
- Do this from now on for the next steps in release management (create release candidate etc.)
Test instabilities:
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31036
  : Blocker
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-18356
  : Starts failing more regularly again. Qingsheng Ren to ping Godfrey He and Yunhong Zheng
- FLINK-30972 (openssl version update necessary) continues to fail because ofFLINK-30965 (repo-sync doesn't pick up 1.15 changes anymore)

2023-02-21

Open Blocker issues
Example Release Jira issues:
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-31146
>
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-31154
Release testing not finished, yet
Decision on RC creation moved to next week's Flink release sync call
Other issues:
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31133
  : Issue in 1.15.3 unexplained

- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31092
  : 1.17 issue in Hive
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-30733
  : no updates on the Slack bot instability so far
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31134
  : OOM issue - waiting for Gabor's response
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31145
  (Kafka infrastructure umbrella ticket)

2023-02-28

Blockers: are these really blockers or do we want to move the release forward?
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31092
  → Qingsheng Ren to check the status with Shengkai Fang
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31104
  → Qingsheng Ren to check the status with Shengkai Fang
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-30978
  → Qingsheng Ren to check the status with Shengkai Fang
Other issues:
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31134
  : Revisit OOM in Kafka e2e test in 1.15.3 - we're keeping it as critical for now. There are no other artifacts that we can investigate. The test doesn't involve table-runner code (therefore, isn't connected to
  Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-18356
  ). → Martijn Visser to check internally
- Dependabot alerts are affecting all commiters now
Release managers to review draft of 1.17 announcement
Release 1.17 preparation:
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-31146

2023-03-07

Ubuntu mirror instabilities:
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-30921
Matthias Pohl will follow-up on that one
API backwards compatibility:
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-31167
Do we have additional documentation on API compatibility? Leonard Xu will do another pass over the ticket comments to confirm the findings
Announcement: FLIPs missing: FLIP-272: Generalized delegation token support, FLIP-217: Support watermark alignment of source splits Leonard Xu reaches out to contributors to add something related to the announcement
Instabilities (blocker query):
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31351
  :Leonard Xu has pinged luoyuxia to look into this issue.
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31278
  : Downgraded→ [Critical] PR ready, flink-runtime tests will be made sequential ,
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31339
  : Downgraded→ [Critical] Leonard Xu has pinged Godfrey He to look into this issue.
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31342
  : Downgraded→ [Critical]
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31341
  : Downgraded→ [Critical]
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31354
  : Downgraded→ [Critical] Matthias Pohl
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31355
  : Downgraded→ [Critical] Leonard Xu has pinged Dian Fu to look into this issue.

2023-03-14

Flink RC2 for 1.17 is available, needs checkers

2023-03-21

RC3 for 1.17.0 was created and is in voting stage (1 binding vote miss)
Robert Metzger agreed to assist with the license check

2023-03-23

Flink 1.17.0 is officially released today!

Retrospective:

From Qingsheng Ren

As discussed in the mailing list, we need to trigger a final patch version for 1.15 after releasing 1.17. Some cleanup steps need to be reviewed and changed, such as removing 1.15 data from svn, CI, flink-docker etc. See
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-31570
I like the idea made by Matthias Pohl that we track TODOs for releasing on JIRA 👍 I used it as a checklist to make sure we don't miss anything. Also it helps collaborating, as we can divide works across RMs easily by assigning JIRA tickets.

From Matthias Pohl

Google Meet might not be the best choice for the release sync. We need to be able to invite attendees even if the creator of the meeting isn't available (maybe try Zoom or even Jitsi as an OpenSource alternative instead?)
Release sync every 2 weeks and a switch to weekly after feature freeze felt reasonable
Slack worked well as a collaboration tool to document the monitoring tasks (#builds, #flink-dev-benchmarks) in a team with multiple release managers
The Slack Azure Pipeline bot seems to be buggy. It swallows some build failures. It's not a severe issue, though. We created #builds-debug to monitor whether it's happening consistently. The issue is covered in
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-30733
We experienced occasional issues in the manual steps of the release creation in the past (e.g. japicmp config was not properly pushed). Creating Jira issues for the release helped to make the release creation more transparent and made the steps more reviewable. Additionally, it helped to distribute subtasks to different people with Jira being the tool for documentation and synchronization. That's especially helpful when there is more than one person in charge of creating the release.
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31146
- RCs
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-31154
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-31578
  - Jira
    server ASF JIRA
    serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
    key FLINK-31583
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31562
- Jira
  server ASF JIRA
  serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
  key FLINK-31567
We had backports/merges without PRs happening by committers occasionally during the 1.17 release which broke master/release branches (probably, changes were done locally before merging which were not part of the PR to have a faster backport experience). It might make sense to remind everyone that this should be avoided. Not sure whether we want/can restrict that.
We observed a good response on fixing test instabilities by the end of the release cycle but had some long running issues earlier in the cycle which caused extra efforts on the release managers due to reoccurring test failures.
Release testing picked up “slowly”: Initially, we planned 2 weeks for release testing. But there was not really any progress (tickets being created and worked on) in the first week. In the end, we had to extend the phase by another week resulting in 3 instead of 2 weeks of release testing. I guess we could encourage the community to create release testing tasks earlier and label them properly to be able to monitor the effort. That would even enable us to do release testing for a certain feature after the feature is done and not necessarily only at the end of the release cycle.
Manual test data generation is tedious (
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-31593
). But this should be fixed in 1.18 with
Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-27518
being almost done.
We started creating documentation for release management. The goal is to collect what tasks are there to help support a Flink release to encourage newcomers to pick up the task.

From Leonard Xu

We can keep RC0 (a non-votable one) in future releases, as an initial version for developers to validate, so that some issues could be found earlier and avoid repeatedly canceling and re-creating RCs.

From Martijn Visser

We should be more careful for commits without a PR / green CI, which brought some problem at the end of 1.17 release cycle. There might not be possible to totally ban this, but we could give an reminder to committers. 26974 → Xingbo is working on thisHow to have monitoring and quality control for the externalized connectors → Need to have a discussion on the Dev mailing list. Martijn Visser to make a proposal and open a discussion thread on this topic.

Page tree

Versions Compared

Old Version 80

New Version Current

Key

Timeline

Highlight features

List

List

Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-29801

Status / Follow-ups

2022-11-01

2022-11-15

2022-11-29

Status / Follow-ups

2022-11-01

2022-11-15

2022-11-29

2022-12-13

2022-12-27

2023-01-10

2023-01-17

2023-01-24

2023-01-31

2023-02-07

2023-02-14

2023-02-21

2023-02-28

2023-03-07

2023-03-14

2023-03-21

2023-03-23

Retrospective:

Page tree

Page History

Versions Compared

Old Version 80

New Version Current

Key

Timeline

Highlight features

List

List

JiraserverASF JIRAserverId5aa69414-a9e9-3523-82ec-879b028fb15bkeyFLINK-29801

Status / Follow-ups

2022-11-01

2022-11-15

2022-11-29

Status / Follow-ups

2022-11-01

2022-11-15

2022-11-29

2022-12-13

2022-12-27

2023-01-10

2023-01-17

2023-01-24

2023-01-31

2023-02-07

2023-02-14

2023-02-21

2023-02-28

2023-03-07

2023-03-14

2023-03-21

2023-03-23

Retrospective:

Jira
server ASF JIRA
serverId 5aa69414-a9e9-3523-82ec-879b028fb15b
key FLINK-29801