Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Release Dates

  • KIP Freeze: Sep 24, 2018 (a KIP must be accepted by this date in order to be considered for this release)
  • Feature Freeze: Oct 1, 2018 (major features merged & working on stabilization, minor features have PR, release branch cut; anything not in this state will be automatically moved to the next release in JIRA)
  • Code Freeze: Oct 15, 2018 (Tentatively)
  • Release: Oct 29Nov 20, 2018 (Tentatively)


These dates are goals and subject to change, but we expect to stay on the Time Based Release Plan unless unexpected critical issues come up. While the target release date is fixed at ~2w after code freeze, RCs will roll out as needed until the release vote passes.

The release manager is Dong Lin.

Release Features

This is a list of key features/improvements/bugfixes that we can include with release notes (WIP)::

  • Java 11 support
  • Support for Zstandard, which achieves compression comparable to gzip with higher compression and especially decompression speeds (KIP-110)
  • Avoid expiring committed offsets for active consumer group (KIP-211)
  • Provide Intuitive User Timeouts in The Producer (KIP-91)
  • Kafka's replication protocol now supports improved fencing of zombies. Previously, under certain rare conditions, if a broker became partitioned from Zookeeper but not the rest of the cluster, then the logs of replicated partitions could diverge and cause data loss in the worst case (KIP-320).

  • Streams API improvements (KIP-319, KIP-321, KIP-330, KIP-353, KIP-356
  • Improvements of log truncation handling to reduce message duplication and message loss (KIP-320, KIP-359)
  • SASL support improvement (KIP-342, KIP-368)
  • Avoid duplicate message consumption for consumer group that uses Sticky Assignor (KIP-341)
  • Admin script and admin client API improvements to simplify admin operation (KIP-231, KIP-308, KIP-322, KIP-324, KIP-338, KIP-340, KIP-339)Streams API
  • DNS handling improvements (KIP-319235, KIP-321, KIP-328, KIP-330, KIP-353, KIP-356, KIP-372)
  • Avoid expiring committed offsets for active consumer group (KIP-211)
  • Add Codec for ZStandard Compression (KIP-110)
  • Provide Intuitive User Timeouts in The Producer (KIP-91)
  • Bound memory usage in the consumer (KIP-81)
  • 302)

How to Contribute

Before code freeze:

  • Participate in votes and discussions to land or postpone the open KIPs.
  • Review patches. We  We anticipate that this release will be bottlenecked mostly on reviews. The more reviewers, the more content we can fit in.
  • Write unit/integration/system tests. We  We want to preserve the tradition of high-quality releases in Apache Kafka. 

After code freeze:

  • Write more unitmore unit/integration/system testsWe want to preserve the tradition of high-quality releases in Apache Kafka.
  • Improve documentation
  • Test the release candidates
  • Open blocker JIRAs on critical issues found. Open non-blocker JIRAs on any other issues found.
  • Fix critical bugs
  • Review bug fixes
  • Vote on RCs. Even though only PMC votes are binding, community votes are super important as we evaluate the readiness of the release

Open Issues

Jira
serverASF JIRA
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
maximumIssues20
jqlQueryproject = KAFKA AND fixVersion = 2.1.0 AND status not in (resolved, closed) ORDER BY priority DESC, status DESC, updated DESC
serverId5aa69414-a9e9-3523-82ec-879b028fb15b

Also feel free to refer to this release page for more details of the included tickets.

Planned KIP Content

Note: The planned content is not binding - final content will be based the features committed by branch-cutting date. See Kafka Improvement Proposals for the full list of KIPs.

KIPJIRAComponentCommitter (reviewer)
KIP-81: Bound Fetch memory usage in the consumer

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-4133

Clients
Comment

KIP-91 Provide Intuitive User Timeouts in The Producer

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-5886

Clients
KIP-110: Add Codec for ZStandard Compression

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-4514

Broker, Clients
KIP-206: Add support for UUID serialization and deserialization

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-4932

Clients
KIP-211: Revise Expiration Semantics of Consumer Group Offsets

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-4682

Broker, Clients
KIP-231: Improve the Required ACL of ListGroups API

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-5638

Security
KIP-
258: Allow to Store Record Timestamps in RocksDB
235: Add DNS alias support for secured connection

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-

3522

6195

Streams
Clients
Matthias Sax

KIP-
289: Improve the default group id behavior in KafkaConsumer

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-6774

ClientsKIP-
302 - Enable Kafka clients to use all DNS resolved IP addresses

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-6863

Clients
KIP-306: Configuration for Delaying Response to Failed Client Authentication

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-6950

Client
KIP-308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides configs

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-6751

Broker
KIP-312: Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7027

Streams
KIP-
313: Add KStream.flatTransform and KStream.flatTransformValues

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-4217

StreamsKIP-
319: Replace segments with segmentInterval in WindowBytesStoreSupplier

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7080

Streams
KIP
-320: Allow fetchers to detect and handle log truncation JiraserverASF JIRAserverId5aa69414-a9e9-3523
-
82ec-879b028fb15bkeyKAFKA-6880Broker,ClientsKIP-
321: Update TopologyDescription to better represent Source and Sink Nodes

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-6966

Streams
KIP-322: Return new error code for DeleteTopics API when topic deletion disabled

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-4690

Tools
KIP-324: Add method to get metrics() in AdminClient

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-6986

AdminClient
KIP-328: Ability to suppress updates for KTables

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-6556

StreamsMatthias Saxpartially implemented in v2.1
KIP-330: Add retentionPeriod in SessionBytesStoreSupplier

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7101

Streams
KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde JiraserverASF JIRAserverId5aa69414-a9e9-3523-82ec-879b028fb15bkeyKAFKA-6161

Core, Streams

KIP-332: Update AclCommand to use AdminClient API

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7117

Tools
KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-6923

Clients
KIP-338 Support to exclude the internal topics in kafka-topics.sh command

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7139

Tools
KIP-339: Create a new IncrementalAlterConfigs APIAminClient


KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7147

Tools
KIP-341: Update Sticky Assignor's User Data Protocol

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7026

Clients


KIP-342: Add support for Custom SASL extensions in OAuthBearer authentication

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7169

Security
KIP-353: Improve Kafka Streams Timestamp Synchronization

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-3514

Streams
KIP-356: Add withCachingDisabled() to StoreBuilder

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-6998

StreamsMatthias Sax
KIP-357: Add support to list ACLs per principal

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-5690

Tools
KIP-358: Migrate Streams API to Duration instead of long ms times

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7277
Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7477

Streams
KIP-
359: Verify leader epoch in produce requests
365: Materialized, Serialized, Joined, Consumed and Produced with implicit Serde

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7396

Streams
KIP-366: Make FunctionConversions deprecated

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-

7383

7399

Streams
Broker,Clients


KIP-
361: Add Consumer Configuration to Disable Auto Topic Creation
372: Naming Repartition Topics for Joins and Grouping

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7406

Streams

 

Postponed to subsequent release

 

KIPJIRAComponentCommitterComment
KIP-81: Bound Fetch memory usage in the consumer

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-

...

...

149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-

...

4218

Streams

...


partially implemented in v1.1
KIP

...

-150 - Kafka-Streams Cogroup

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-

...

...

6049

Streams

KIP-289: Improve the default group id behavior in KafkaConsumer

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-6774

Clients

KIP-313: Add KStream.flatTransform and KStream.flatTransformValues

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-

...

4217

Streams

...

...

...

fetchers to detect and handle log truncation

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-

...

6880

Broker,Clients
KIP-

...

328: Ability to suppress updates for KTables

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-

...

6556

Streams

 

Postponed to subsequent release

partially implemented in v2.1
KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-6161

Core, Streams

KIP-339: Create a new IncrementalAlterConfigs API

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7466

AminClient
KIP-341: Update Sticky Assignor's User Data Protocol

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7026

Clients
KIP-359: Verify leader epoch in produce requests

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7383

Broker,Clients

KIP-361: Add Consumer Configuration to Disable Auto Topic Creation

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-7320

Clients

KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)

 

KIPJIRAComponentCommitter
KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-42187391

StreamsClients,AdminClient

KIP-150 - Kafka-Streams Cogroup368: Allow SASL Connections to Periodically Re-Authenticate

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyKAFKA-60497352

Broker,ClientsStreams