This Wiki page is to coordinate the testing efforts in the community. For the community testing, the feature authors are ideally not testing their own features. They are just providing minimal pointers to the docs or configuration parameters, so that we can test the usability of the new feature as well.

Release Testing Status

Current Release Candidate: RC2 (December 1st, 2020)

Binaries: https://dist.apache.org/repos/dist/dev/flink/flink-1.12.0-rc2/

Maven Staging: https://repository.apache.org/content/repositories/orgapacheflink-1403/

Flink 1.12 Burndown (blockers and critical bugs): https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=406&projectKey=FLINK

Currently known major bugs:

  • New Kafka Source is not working ( Unable to render Jira issues macro, execution error. ) - present in RC2, the Kafka source will not be announced as a new feature.

Testing Topics

When testing a feature, consider the following aspects:

  • Is the documentation easy to understand
  • Are the error messages, log messages, APIs etc. easy to understand
  • Is the feature working as expected under normal conditions
  • Is the feature working / failing as expected with invalid input, induced errors etc.


Topic
(+ Feature Ticket)

Tested by

(Please state your name, and the RC you've used)

Testing Ticket

Issues foundNotes

K8s High Availability Service

Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.



Kafka Source based on the new Source API  Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.



Batch execution for the DataStream API  Unable to render Jira issues macro, execution error. / Unable to render Jira issues macro, execution error. / Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.


Note to testers: The three issues can really only be tested in combination. FLINK-19316 is done but missing documentation.

  • Write an example that uses a (new) FileSource, a (new) FileSink, some random transformations
  • Run the example in BATCH mode
  • How ergonomic is the API/configuration?
  • Are there any weird log messages/exceptions in the JM/TM logs
  • Maybe try sth that doesn't work on BATCH execution, such as iterations/feedback edges.

Intra-Slot Managed Memory Sharing

Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.



Flink on Mesos Deployment

Unable to render Jira issues macro, execution error.

Matthias

Unable to render Jira issues macro, execution error.


This is not a new feature. We have touched the Mesos deployment during the ResourceManager refactor. Would be helpful verify the deployment with manual tests.

New File Source API - Continuous Streaming Execution

Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.



FLIP-132 Temporal Table DDL and Temporal Table Join

Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.



FLIP-107: Handling of metadata in SQL connectors

Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.



FLIP-149: Introduce the upsert-kafka Connector

Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.



Introduce multi-input operator

Unable to render Jira issues macro, execution error.



Note to testers: check if log and Web UI is showing meaningful names of tasks with chained sources.

Temporal Join on Hive table

Unable to render Jira issues macro, execution error.




Small file compaction

Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.



Pandas UDAF in Python Table API
Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.



Python UDAF in Python Table API
Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.



Python DataStream API
Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.



Python workers resource management
Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.



Native support of PyFlink on Kubernetes

Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.



Support collect in the Python Table API
Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.



Non-blocking job submission

Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.


Test proper error reporting in case JobManager initialization hangs

FLIP-119: Pipelined region scheduler

Unable to render Jira issues macro, execution error.

Robert (in progress)

Unable to render Jira issues macro, execution error.


high parallelism; many failures; many regions; large cluster;

Displaying memory configuration in the web ui

Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.


The following questions should be answered for the different deployment modes (application cluster, per-Job cluster, session cluster) and deployment backends (Standalone/Local, Kubernetes, Yarn, Mesos):

  • Does the TaskManager metrics page (path: /#/task-manager/<taskmanager-id>/metrics) looks as proposed in FLIP-102 (minor improvements where introduced in  Unable to render Jira issues macro, execution error. )? It helps to have a job running to see whether the metrics in the right column are actually dynamically changing.

  • Check whether all 8 newly introduced metrics are exposed through the TaskManager's metric endpoint:

    Metrics path 
    http://localhost:8081/taskmanagers/<taskmanager-id>/metrics?get=Status.Flink.Memory.Managed.Total,Status.Flink.Memory.Managed.Used,Status.JVM.Memory.Metaspace.Committed,Status.JVM.Memory.Metaspace.Used,Status.JVM.Memory.Metaspace.Total,Status.Shuffle.Netty.UsedMemory,Status.Shuffle.Netty.AvailableMemory,Status.Shuffle.Netty.TotalMemory 
  • Does the JobManager metrics page (path: /#/job-manager/metrics) as proposed in FLIP-104?
  • Does the Cluster configuration endpoint include /#/job-manager/config include memory-related parameters like jobmanager.memory.heap.size, jobmanager.memory.jvm-metaspace.size, or jobmanager.memory.off-heap.size? Do they match configuration parameters possibly set in the config file?

Improved unaligned checkpointing times under backpressure

Unable to render Jira issues macro, execution error.

Arvid

Unable to render Jira issues macro, execution error.



Test SQL Hive integration

Unable to render Jira issues macro, execution error.


Test how easy it is to use Flink's Hive integration. In particular pay attention to

  • Dependency management is easy (no unclear dependencies)
  • Configuration works smoothly (no relative file paths in configuration)
  • No labels