You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 95 Next »

Table of contents

Jira Boards

Flink 1.17 Burndown: https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=572

Sync meeting

The sync meeting is happening every second Tuesday starting on the 1st of November 2022 at 9am CEST / 4pm China Standard Time / 8am UTC. 
As we are getting closer to the feature freeze we will do the meeting on a weekly base.


Feel free to join on Google Meet. Local dial-in numbers can be found at https://tel.meet/wcx-fjbt-hhz?pin=1940846765126

Timeline

Feature Freeze
January, 17th of 2023, end of business CEST. 

Chinese new year is 22 January: 1+ week holiday. Ppl will be back in early February

Release
Mid of March 2023

Highlight features

please feel free to add/suggest.

Features

List of features announced by contributors and committers that are likely to be ready for the feature freeze:n

NOTICE: It's preferred if only new features end up there and not all bugs/tasks separately, so that the page is not over bloated. Of course, unless fixing a bug is a really big or important one equivalent to implementing a completely new feature. A good rule of thumb would be that each entry in the page could (but does not have to) be later on included in a release blog post.

Legend

State

symbolmeaningcomment
(big grin)validatedthrough cross team testing
(tick)donewell documented with a complete test coverage
(green star)will make itthere is no reason this effort should not go into 1.17
(star)in dangerthere are some concerns the effort could be ready for the feature freeze of 1.17
(red star)very unlikelythere are severe concerns the effort could make it to 1.17
(minus)won't make itit was decided against adding this for the 1.17 release. working on the effort has been stopped.
(question)state unclear
(blue star)independentas the artefact could be released independent of Apache Flink

X-Team verification

symbolmeaning
(tick)done
(blue star)not required

Feature Stage

Please align with the list on the Apache Flink Roadmap (https://flink.apache.org/roadmap.html).

  • MVP: Have a look, consider whether this can help you in the future.
  • Beta: You can benefit from this, but you should carefully evaluate the feature.
  • Ready and Evolving: Ready to use in production, but be aware you may need to make some adjustments to your application and setup in the future, when you upgrade Flink.
  • Stable: Unrestricted use in production
  • Reaching End-of-Life: Stable, still feel free to use, but think about alternatives. Not a good match for new long-lived projects.
  • Deprecated: Start looking for alternatives now 

Summary

Numbers are based on the items in the list below, not on the tickets 


(tick) (green star) (star)(red star)(minus)(question)(blue star)Remaining weeks
2022-11-01







11
2022-11-15







0
2022-11-29410000230377
2022-12-1372410190425
2022-12-2772510180423
2023-01-03







2
2023-01-10







1
2023-01-17







0

List

Feel free to add categories.

Runtime
Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified

































Shuffle









Unable to render Jira issues macro, execution error.

Xintong Song 

(tick)100%10-01-2022


Adaptive Batch Scheduler supports hybrid shuffle modeWeijie Guo Xintong Song 

(minus)




Unable to render Jira issues macro, execution error.

Yuxin Tan Xintong Song 

(green star)50%09-01-2023


Further improvement of production availability of hybrid shuffle

(green star)50%10-01-2022


Deployment & Cluster Coordination
Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified
REST API

Unable to render Jira issues macro, execution error.




(tick)
15-11-202215-11-202215-11-2022

Unable to render Jira issues macro, execution error.


The first stage is finished, and the second stage is improvement, it will be finished in the next version.(tick)100%09-01-202305-01-202309-01-2023
Reactive Mode











Scheduler

Unable to render Jira issues macro, execution error.



(green star)80%03-01-2022


FLIP-281: Speculative execution for sinkBiao Liu 
FLIP in discussion(green star)30%03-01-2022


FLIP-283: Use adaptive batch scheduler as default scheduler for batch jobs
FLIP in discussion(green star)30%03-01-2023













Misc

































OLAP






















State backend
Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified

FLIP-263: Improve resolving schema compatibility

Unable to render Jira issues macro, execution error.

Hangxiang Yu 

(green star)80%10-1-2022


Improve File Management in State Backend




(green star)5%13-12-2022


Improve the serializer performace of state change of changelog

Unable to render Jira issues macro, execution error.



(green star)100%10-1-2022


Allow to configure Changelog Storage per program

Unable to render Jira issues macro, execution error.



(green star)50%13-12-2022


Add a metric for back-pressure from the ChangelogStateBackend

Unable to render Jira issues macro, execution error.



(green star)40%13-12-2022


Frocksdb cannot run on Apple M1

Unable to render Jira issues macro, execution error.



(green star)90%13-12-2022


Checkpoint
Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified
Benchmark Generic Incremental CP + UC + BDYuan Mei 

(question)




Unable to render Jira issues macro, execution error.



(green star)70%10-12-2022


Benchmark









Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified

Improve benchmark stability

Unable to render Jira issues macro, execution error.

Yuan Mei 

(green star)50%3-1-2023















































































API
Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified

Unable to render Jira issues macro, execution error.


Deprecated
(tick)100%08-11-202208-11-202208-11-2022

Unable to render Jira issues macro, execution error.


Removed
(green star)40%08-11-2022


Unable to render Jira issues macro, execution error.





(question)




Unable to render Jira issues macro, execution error.


FLIP Discussion
(question)
13-12-2022


Unable to render Jira issues macro, execution error.

Martijn Visser 
Removed
(tick)100%14-11-202214-11-202214-11-2022

Unable to render Jira issues macro, execution error.





(question)




SQL
Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified
General

Unable to render Jira issues macro, execution error.

Godfrey He ,@Yunhong Zheng




(green star)70%2023-01-10


Unable to render Jira issues macro, execution error.




(green star)




Unable to render Jira issues macro, execution error.

Chesnay Schepler 


(green star)




Calcite Update

Unable to render Jira issues macro, execution error.




(tick)




Unable to render Jira issues macro, execution error.




(tick)




Unable to render Jira issues macro, execution error.

Sergey Nuyanzin 


(green star)




Flink Dialect

ALTER TABLE API

Unable to render Jira issues macro, execution error.




(green star)90%2023-01-10


DELETE/UPDATE API (FLIP-282) 

voting(green star)
2023-01-10


Hive Dialect

Unable to render Jira issues macro, execution error.




(green star)60%



Unable to render Jira issues macro, execution error.




(green star)80%2023-01-10


Unable to render Jira issues macro, execution error.




(question)




SQL Gateway

Unable to render Jira issues macro, execution error.

Shengkai Fang  , Zelin Yu




(green star)70%2023-01-10













QE
FLIP-280: Introduce a new explain mode to provide SQL advice

voting(green star)
2023-01-10


























































































QO

FLINK-27591 - Improve the plan for batch queries when statistics is unavailable OPEN

Godfrey He ,@Yunhong Zheng





(green star)40%






































































































API/Python
Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified

Unable to render Jira issues macro, execution error.

HuangXingbo 


(tick)100%12-12-2022


Unable to render Jira issues macro, execution error.

HuangXingbo 


(star)0%27-12-2022


Unable to render Jira issues macro, execution error.

HuangXingbo 


(green star)70%27-12-2022


Unable to render Jira issues macro, execution error.

HuangXingbo 


(green star)0%27-12-2022


Unable to render Jira issues macro, execution error.

HuangXingbo 


(tick)100%12-12-2022


Machine Learning
Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified











CEP
Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified












































Web
Name/JIRA IssueResponsible ContributorReviewer / committer availableFeature StageNoteState%UpdatedImplementedDocumentedX-team verified

Unable to render Jira issues macro, execution error.



(question)
















































Status / Follow-ups

2022-11-01

Agenda

  • Kickoff
  • Keeping the state of features updated (ideally before the sync)
  • Blockers
    • Unable to render Jira issues macro, execution error.
  • Build stability
  • Are there any (new) contributors who need a PR reviewed or merged? And if yes, who can help out?

2022-11-15

Agenda

  • Kickoff
  • Keeping the state of features updated (ideally before the sync)
  • Blockers
    • Unable to render Jira issues macro, execution error.
    • Unable to render Jira issues macro, execution error.
    • Unable to render Jira issues macro, execution error.
  • Build stability
    • Number of growing test stability issues with "Exit code 137 errors"
      • Unable to render Jira issues macro, execution error.
      • Unable to render Jira issues macro, execution error.
      • Unable to render Jira issues macro, execution error.
  • Are there any (new) contributors who need a PR reviewed or merged? And if yes, who can help out?

2022-11-29

  • Build instabilities
    • Unable to render Jira issues macro, execution error. : Anton has some new findings on that issue and will get back on it
    • Pulsar-related issues:
      • Unable to render Jira issues macro, execution error.
      • Unable to render Jira issues macro, execution error.
      • Unable to render Jira issues macro, execution error.
    • Python-related issues:
    • Unable to render Jira issues macro, execution error. : OOM errors are most likely being caused by flink-table-planner (Godfrey He might be a person to reach out to about it)
    • Unable to render Jira issues macro, execution error. : Qingsheng Ren will look into the PR
    • Unable to render Jira issues macro, execution error. Matthias Pohl will ping contributors on the issue - but it's not that urgent since it's not failing that frequently 
  • Other topics:
    • Externalizing Pulsar connector (test instabilities): Martijn Visser  is working on externalizing the code base
    • Performance test monitoring: Discussion is happening on the mailing list
    • Externalizing connectors in general is work-in-progress but looks good
    • Public CI documentation can be improved

2022-12-13

  • Build instabilities
    • Unable to render Jira issues macro, execution error. → Qingsheng to have a look at the PR
    • Pulsar connector has been synced to external connector repository. Martijn Visser to open a PR to remove connector from master
    • Unable to render Jira issues macro, execution error. → Qingsheng to ping Godfrey
    • Unable to render Jira issues macro, execution error. → The PR is still failing for the same issue as this PR should fix. We should ping the author to have a look first
    • Unable to render Jira issues macro, execution error. → Martijn to ping Thomas once more
    • Unable to render Jira issues macro, execution error. → Xingbo is working on this
  • How to have monitoring and quality control for the externalized connectors → Need to have a discussion on the Dev mailing list. Martijn Visser to make a proposal and open a discussion thread on this topic.

2022-12-27

  • Meeting skipped due to Christmas holiday/sick leaves
  • Discussion started on moving the feature freeze from Jan 17 to Jan 31 due to pandemic situation in China (see dev ML discussion thread)
  • Pulsar connector has been externalized. Pulsar-related test instabilities were disabled in release-1.16  and release-1.15  (see Unable to render Jira issues macro, execution error. and parent task)

2023-01-10

  • Build instabilities (all 1.17 test instabilities with a priority >=Major)
    • Unable to render Jira issues macro, execution error.
    • Unable to render Jira issues macro, execution error.
    • Unable to render Jira issues macro, execution error.
  • Proposal: Priorities of test instabilities (docs about it)
    • Blocker: New test instabilities which were introduced in unreleased versions only
    • Critical: New test instabilities that also affect released versions; Test instabilities that pop up more than two times per month (because they affect CI maintenance)
    • Major: Test instabilities that popped up after the version was released but happen rarely
  • Switching to weekly calls as we're getting closer to the feature freeze?



  • No labels