This document is an in progress loose collection of best practices for adding code to Flink and lessons learned from past contributions. These are not enforced in any kind yet and some points might require more detailed explanations about why and when to apply them. In the future, we might distill a set of common rules from these and be more strict about applying them. For now, this is an in progress list that you should feel free to extend.

General

• Always add comments when you deprecate something

• Add a @Deprecated annotation to the JavaDocs explaining why it was deprecated (removed, replaced, etc.)

• Create issues with target release version for deleting deprecated interfaces

• https://docs.oracle.com/javase/1.5.0/docs/guide/javadoc/deprecation/deprecation.html

• Use annotations in order to help IDE static analysis

  • @GuardedBy (javax.annotation.concurrent)

  • @Nullable (javax.annotation)
• Add assert Thread.holdsLock(lock) when you expect to hold a lock
  

  • Costs nothing at runtime (assertions are disabled), but helps in tests (assertions are enabled)
    

  • https://docs.oracle.com/javase/8/docs/technotes/guides/language/assert.html

•  Continue disposal operations if one sub operation fails
  • Example: MiniCluster#shutdownInternally() (https://github.com/apache/flink/blob/51a3573/flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java#L291)

    • Shuts down many things each of which can throw Exceptions

    • Use Java 7 suppressed Exceptions (https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html#suppressed-exceptions)

    • Flink Utilities

      • ExceptionUtils.firstOrSuppressed

      • ExceptionUtils.rethrowException

• When statically importing static members of other classes in main code, it can help readability to specify the full class name, e.g. ExceptionsUtils.rethrowException(Throwable) instead of importing statically and only calling rethrowException(Throwable).

• In can be inefficient to use in-place anonymous subclasses where static can be used

• In loops this is an easy win to avoid creating objects

• If it’s only called a few times it is OK

• There is an IntelliJ inspection for creating many objects in loops

• Example: MergingWindowSet (https://github.com/apache/flink/blob/51a3573/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/windowing/MergingWindowSet.java#L154)

• All classes in flink-core and other public projects need to have visibility annotations (@Public, @PublicEvolving, @Internal)

  • https://cwiki.apache.org/confluence/display/FLINK/Stability+Annotations

• If functions execute typically small parts of the logic, move other parts out of the function

• KeyedStateBackend#mergedPartitionedStates (https://github.com/apache/flink/blob/51a3573/flink-runtime/src/main/java/org/apache/flink/runtime/state/AbstractKeyedStateBackend.java#L295)

• Very long functions with very few lines that are relevant

• Not good for performance on the code cache locality side

• Consecutive code, but we have to jump over large areas of the code

• Put code blocks into private functions

• The function holding it together (if-elseif-else) is compact and jumps to other parts that are also compact

• Smaller methods usually better for inlining

•   Use System.nanoTime when measuring times, because   System.currentTimeMillis is not stable
  

Tests

• Ignored tests should always have a comment explaining why it’s ignored. Otherwise it’s hard to keep track of why tests were ignored or if they were relevant.

• Also, add a JIRA issue to unignore that test.

• Avoid sleeps in tests

  • Build tests on latches and blocking queues

  • CheckpointCoordinatorTest#testMinTimeBetweenCheckpointsInterval() (https://github.com/apache/flink/blob/51a357351b955844941edd9a9b1406cdc787b18a/flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinatorTest.java#L1222)

• Use org.apache.flink.core.testutils.CheckedThread if you want to catch Exceptions in threads or use an ExecutorService and check the Future result.
  

IDE Setup

• Have a look at your IDE settings and activate more inspections, especially around generic types, constant expressions, serialization.