List of API breaking changes

• 3c236b2c89ed84cded0d056ac40fc728089c6f67 changes the signature of the Scala DataSet API's ExecutionEnvironment.fromCollection() method and breaks binary compatibility.
• 06b37bf550315bd1d5be7dc3ed6638fd21768e1a removes JDBC Input/OutputFormats for the deprecated Record API
  
• bac21bf5d77c8e15c608ecbf006d29e7af1dd68a:
  • Return value of addSink in DataStream API is no longer a DataStream but a DataStreamSink
  • Forward shipping is not legal when the parallelism of the operations differ
• e727355e42bd0ad7d403aee703aaf33a68a839d2  (FLINK-2723) CopyableValue interface has a new method "T copy()".
  
  

TO BE FIXED

BLOCKER

FLINK-2747 TypeExtractor does not correctly analyze Scala Immutables

Solution:

FLINK-2236 RowSerializer and CaseClassComparator are not in sync regarding Null-Values

Solution: revert previous changes to the master OR fix

FLINK-2066 Make delay between execution retries configurable

Solution: Pending pull request

CRITICAL

FLINK-2674 Rework windowing logic

FLINK-2674 Rework windowing logic

FLINK-2491 Operators are not participating in state checkpointing in some cases

MAJOR

ALL OPEN ISSUES

BLOCKER

FLINK-2747 TypeExtractor does not correctly analyze Scala Immutables (AnyVal)

FLINK-2236 RowSerializer and CaseClassComparator are not in sync regarding Null-Values [fix or revert]

FLINK-2066 Make delay between execution retries configurable [pending pull request]

CRITICAL

FLINK-2674 Rework windowing logic

FLINK-2771 IterateTest.testSimpleIteration fails on Travis

FLINK-2770 KafkaITCase.testConcurrentProducerConsumerTopology fails

FLINK-2719 ProcessFailureStreamingRecoveryITCase>AbstractProcessFailureRecoveryTest.testTaskManagerProcessFailure failed on Travis

FLINK-2695 KafkaITCase.testConcurrentProducerConsumerTopology failed on Travis

FLINK-2674 Rework windowing logic

FLINK-2671 Instable Test StreamCheckpointNotifierITCase

FLINK-2670 Unstable CombineTaskTest

FLINK-2586 Unstable Storm Compatibility Tests

FLINK-2504 ExternalSortLargeRecordsITCase.testSortWithLongAndShortRecordsMixed failed spuriously

FLINK-2491 Operators are not participating in state checkpointing in some cases

FLINK-2392 Instable test in flink-yarn-tests

FLINK-2348 IterateExampleITCase failing

MAJOR

FLINK-2609 Automatic type registration is only called from the batch execution environment

FLINK-2797 CLI: Missing option to submit jobs in detached mode

FLINK-2749 CliFrontendAddressConfigurationTest fails on GCE

FLINK-2319 Collect size limitation due to akka.

FLINK-2662 CompilerException: "Bug: Plan generation for Unions picked a ship strategy between binary plan operators."

FLINK-2508 Confusing sharing of StreamExecutionEnvironment

FLINK-2757 DataSinkTaskTest fails on Windows

FLINK-2351 Deprecate config builders in InputFormats and Output formats

FLINK-2350 Deprecate the stream open timeouts in FileInputFormat and FileOutputFormat

FLINK-2794 Document exactly-once support for sources and sinks

FLINK-2813 Document off-heap configuration

FLINK-2263 ExecutionGraph uses Thread.sleep to delay execution retries

FLINK-2750 FileStateHandleTest fails when building for Hadoop 2.6.0

FLINK-2657 Fix WebClient and CLI interaction

FLINK-2506 HBase connection closing down (table distributed over more than 1 region server - Flink Cluster-Mode)

FLINK-2315 Hadoop Writables cannot exploit implementing NormalizableKey

FLINK-1610 Java docs do not build

FLINK-2735 KafkaProducerITCase.testCustomPartitioning sporadically fails

FLINK-2737 KeyedDataStream should not be a subclass of DataStream

FLINK-2718 LeaserChangeCleanupTest sometimes fails

FLINK-2624 RabbitMQ source / sink should participate in checkpointing

FLINK-2801 Rework Storm Compatibility Tests

FLINK-2156 Scala modules cannot create logging file

FLINK-2408 Some maven properties are defined in build profiles only

FLINK-1989 Sorting of POJO data set from TableEnv yields NotSerializableException

FLINK-2759 TaskManagerProcessReapingTest fails on Windows

FLINK-2758 TaskManagerRegistrationTest fails on Windows

FLINK-1129 The Plan Visualizer Cuts of the Lower Part of Certain Operators

FLINK-2611 YARN reports failed final state for successful jobs

FLINK-2799 Yarn tests cannot be executed with DEBUG log level

FLINK-2733 ZooKeeperLeaderElectionTest.testZooKeeperReelection fails

FLINK-2443 [CompactingHashTable] GSA Connected Components fails with NPE

FLINK-2524 Add "getTaskNameWithSubtasks()" to RuntimeContext

FLINK-2767 Add support Scala 2.11 to Scala shell

FLINK-2706 Add support for streaming RollingFileSink to truncate / append on UNIX file systems

FLINK-2788 Add type hint with TypeExtactor call on Hint Type

FLINK-2581 Add warning to the YARN client when using an older Hadoop version

FLINK-2518 Avoid predetermination of ports for network services

FLINK-2344 Deprecate/Remove the old Pact Pair type

FLINK-987 Extend TypeSerializers and -Comparators to work directly on Memory Segments

FLINK-2256 Handling Broker failures on the KafkaSink side

FLINK-2287 Implement JobManager high availability

FLINK-1778 Improve normalized keys in composite key case

FLINK-2489 Improve the streaming window join

FLINK-2523 Make task canceling interrupt interval configurable

FLINK-2455 Misleading I/O manager error log messages

FLINK-2795 Print JobExecutionResult for interactively invoked jobs

FLINK-2340 Provide standalone mode for web interface of the JobManager

FLINK-2629 Refactor YARN ApplicationMasterActor to use the async AM<-->RM client

FLINK-2782 Remove deprecated features for 0.10

FLINK-2808 Rework / Extend the StatehandleProvider

FLINK-2667 Rework configuration parameters

FLINK-2403 Rework partitioned state checkpointing logic

FLINK-2646 Rich functions should provide a method "closeAfterFailure()"

FLINK-2075 Shade akka and protobuf dependencies away

FLINK-2546 The webclient does not allow users to save the execution plan

FLINK-2239 print() on DataSet: stream results and print incrementally

FLINK-2254 Add Bipartite Graph Support for Gelly

FLINK-2720 Add Storm-CountMetric in flink-stormcompatibility

FLINK-1723 Add cross validation for model evaluation

FLINK-2728 Add missing features to new web dashboard

FLINK-2435 Add support for custom CSV field parsers

FLINK-2157 Create evaluation framework for ML library

FLINK-2390 Replace iteration timeout with algorithm for detecting termination

FLINK-2333 Stream Data Sink that periodically rolls files

FLINK-2352 [Graph Visualization] Integrate Gelly with Gephi

FLINK-2730 FLINK-2728 Add CPU/Network utilization graphs to new web dashboard

FLINK-2731 FLINK-2728 Add JobManager log file access

FLINK-2819 FLINK-2674 Add Windowed Join/CoGroup Operator Based on Tagged Union

FLINK-2676 FLINK-2674 Add abstraction for keyed window state

FLINK-2732 FLINK-2728 Add access to the TaskManagers' log file and out file in the web dashboard.

FLINK-2790 FLINK-2287 Add high availability support for Yarn

FLINK-2811 FLINK-2728 Add page with configuration overview

FLINK-2680 FLINK-2674 Create a dedicated aligned-event time window operator

FLINK-2354 FLINK-2287 Recover running jobs on JobManager failure

FLINK-2780 FLINK-2674 Remove Old Windowing Logic and API

FLINK-2784 FLINK-2782 Remove deprecated config keys

FLINK-2550 FLINK-2674 Rework DataStream API

FLINK-2779 FLINK-2674 Update documentation to reflect new Stream/Window API

FLINK-2574 Remove Spargel from master in next release

FLINK-2703 Remove log4j classes from fat jar / document how to use Flink with logback

FLINK-2561 Sync Gelly Java and Scala APIs

FLINK-2573 Add Kerberos test case

FLINK-2444 Add tests for HadoopInputFormats

FLINK-2445 Add tests for HadoopOutputFormats

FLINK-2480 Improving tests coverage for org.apache.flink.streaming.api

FLINK-2481 Test at least once processing guarantee for streaming

MINOR

FLINK-2817    FileMonitoring function throws NPE when location is empty

FLINK-2812    KeySelectorUtil.getSelectorForKeys and TypeExtractor.getKeySelectorTypes are incompatible

FLINK-2810    Warn user if bc not installed

FLINK-2803    Add test case for Flink's memory allocation

FLINK-2752    Documentation is not easily differentiable from the Flink homepage

FLINK-2739    Release script depends on the order of parent module information

FLINK-2715    Benchmark Triangle Count methods

FLINK-2672    Add partitioned output format to HDFS RollingSink

FLINK-2634    Add a Vertex-centric Version of the Tringle Count Library Method

FLINK-2613    Print usage information for Scala Shell

FLINK-2608    Arrays.asList(..) does not work with CollectionInputFormat

FLINK-2597    Add a test for Avro-serialized Kafka messages

FLINK-2520    StreamFaultToleranceTestBase does not allow for multiple tests

FLINK-2510    KafkaConnector should access partition metadata from master/cluster

FLINK-2488    Expose attemptNumber in RuntimeContext

FLINK-2429    Remove the "enableCheckpointing()" without interval variant

FLINK-2428    Clean up unused properties in StreamConfig

FLINK-2399    Fail when actor versions don't match

FLINK-2380    Allow to configure default FS for file inputs

FLINK-2342    Add new fit operation and more tests for StandardScaler

FLINK-2283    Make grouped reduce/fold/aggregations stateful using Partitioned state

FLINK-2185    Rework semantics for .setSeed function of SVM

FLINK-1702    Authenticate via Kerberos from the client only

FLINK-1513    Remove GlobalConfiguration Singleton

FLINK-1278    Remove the Record special code paths

FLINK-1240    We cannot use sortGroup on a global reduce