Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Use this number as the identifier for your KIP and increment this value.

Adopted KIPs


KIP

Release

12KIP-366: Make FunctionConversions private2.1.0KIP-359: Verify leader epoch in produce requests2.1.0
32KIP-356: Add withCachingDisabled() to StoreBuilder2.1.0
43KIP-353: Improve Kafka Streams Timestamp Synchronization2.1.0
54KIP-342 Add support for custom SASL extensions in OAuthBearer authentication2.1.0 (WIP)
65KIP-341: Update Sticky Assignor's User Data Protocol2.1.0 (WIP)
76KIP-338 Support to exclude the internal topics in kafka-topics.sh command2.1.0
87KIP-340: Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file2.1.0
98KIP-332: Update AclCommand to use AdminClient API2.1.0 (WIP)
109KIP-330: Add retentionPeriod in SessionBytesStoreSupplier2.1.0
1110KIP-328: Ability to suppress updates for KTables2.1.0
1211KIP-324: Add method to get metrics() in AdminClient2.1.0
1312KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.2.1.0
1413KIP-321: Update TopologyDescription to better represent Source and Sink Nodes2.1.0
1514KIP-320: Allow fetchers to detect and handle log truncation2.1.0
1615KIP-319: Replace segments with segmentInterval in WindowBytesStoreSupplier2.1.0
1716KIP-308: Support dynamic update of max.connections.per.ip/max.connections.per.ip.overrides2.1.0
1817KIP-306: Configuration for Delaying Response to Failed Authentication2.1.0
1918KIP-305: Add Connect primitive number converters2.0.0
2019KIP-303: Add Dynamic Routing in Streams Sink2.0.0
2120KIP-298: Error Handling in Connect2.0.0
2221KIP-297: Externalizing Secrets for Connect Configurations2.0.0
2322KIP-295 Add Streams Configuration Allowing for Optional Topology Optimization2.0.0
2423KIP-294 - Enable TLS hostname verification by default2.0.0
2524KIP-292: Add transformValues() method to KTable2.0.0
2625KIP-290: Support for Prefixed ACLs2.0.0
2726KIP-289: Improve the default group id behavior in KafkaConsumer2.1.0 (WIP)
2827KIP-285: Connect Rest Extension Plugin2.0.0
2928KIP-284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE2.0.0
3029KIP-283: Efficient Memory Usage for Down-Conversion2.0.0
3130KIP-282: Add the listener name to the authentication context2.0.0
3231KIP-281: ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User2.0.0
3332KIP-279: Fix log divergence between leader and follower after fast leader fail over2.0.0
3433KIP-278 - Add version option to Kafka's commands2.0.0
3534KIP-277 - Fine Grained ACL for CreateTopics API2.0.0
3635KIP-276 - Add StreamsConfig prefix for different consumers2.0.0
3736KIP-274: Kafka Streams Skipped Records Metrics2.0.0
3837KIP-272: Add API version tag to broker's RequestsPerSec metric2.0.0
3938KIP-270 - A Scala Wrapper Library for Kafka Streams2.0.0
4039KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade2.0.0
4140KIP-267: Add Processor Unit Test Support to Kafka Streams Test Utils2.0.0
4241KIP-266: Fix consumer indefinite blocking behavior2.0.0
4342KIP-265: Make Windowed Serde to public APIs2.0.0
4443KIP-261: Add Single Value Fetch in Window Stores2.0.0
4544KIP-257 - Configurable Quota Management2.0.0
4645KIP-255: OAuth Authentication via SASL/OAUTHBEARER2.0.0
4746KIP-251: Allow timestamp manipulation in Processor API2.0.0
4847KIP-249: Add Delegation Token Operations to KafkaAdminClient2.0.0
4948KIP-247: Add public test utils for Kafka Streams1.1.0
5049KIP-245: Use Properties instead of StreamsConfig in KafkaStreams constructor2.0.0
5150KIP-244: Add Record Header support to Kafka Streams Processor API2.0.0
5251KIP-243: Make ProducerConfig and ConsumerConfig constructors public1.1.0
5352KIP-239 Add queryableStoreName() to GlobalKTable1.1.0
5453KIP-238: Expose Kafka cluster ID in Connect REST API1.1.0
5554KIP-237: More Controller Health Metrics2.0.0
5655KIP-233: Simplify StreamsBuilder#addGlobalStore1.1.0
5756KIP-231: Improve the Required ACL of ListGroups API2.1.0 (WIP)
5857KIP-229: DeleteGroups API1.1.0
5958KIP-227 - Introduce Incremental FetchRequests to Increase Partition Scalability1.1.0
6059KIP-226 - Dynamic Broker Configuration1.1.0
6160KIP-225 - Use tags for consumer “records.lag” metrics1.1.0
6261KIP-224: Add configuration parameter `retries` to Streams API1.1.0
6362KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer2.0.0
6463KIP-222 - Add Consumer Group operations to Admin API2.0.0
6564KIP-220: Add AdminClient into Kafka Streams' ClientSupplier1.1.0
6665KIP-219 - Improve quota communication2.0.0
6766KIP-218: Make KafkaFuture.Function java 8 lambda compatible1.1.0
6867KIP-215: Add topic regex support for Connect sinks1.1.0
6968KIP-214: Add zookeeper.max.in.flight.requests config to the broker1.1.0
7069KIP-212: Enforce set of legal characters for connector names1.1.0
7170KIP-211: Revise Expiration Semantics of Consumer Group Offsets2.1.0 (WIP)
7271KIP-210 - Provide for custom error handling when Kafka Streams fails to produce1.1.0
7372KIP-208: Add SSL support to Kafka Connect REST interface1.1.0
7473KIP-207: Offsets returned by ListOffsetsResponse should be monotonically increasing even during a partition leader change2.0.0 (WIP)
7574KIP-206: Add support for UUID serialization and deserialization2.1.0 (WIP)
7675KIP-205: Add all() and range() API to ReadOnlyWindowStore1.1.0
7776KIP-204 : Adding records deletion operation to the new Admin Client API1.1.0
7877KIP-203: Add toLowerCase support to sasl.kerberos.principal.to.local rule 1.1.0
7978KIP-202: Move merge() from StreamsBuilder to KStream1.0.0
8079KIP-198: Remove ZK dependency from Streams Reset Tool1.0.0
8180KIP-197 Connect REST API should include the connector type when describing a connector1.0.0
8281KIP-196: Add metrics to Kafka Connect framework1.0.0
8382KIP-195: AdminClient.createPartitions1.0.0
8483KIP-192 : Provide cleaner semantics when idempotence is enabled1.0.0
8584KIP-191: KafkaConsumer.subscribe() overload that takes just Pattern1.0.0
8685KIP-190: Handle client-ids consistently between clients and brokers1.0.0
8786KIP-189: Improve principal builder interface and add support for SASL1.0.0
8887KIP-188 - Add new metrics to support health checks1.0.0
8988KIP-187 - Add cumulative count metric for all Kafka rate metrics1.0.0
9089KIP-186: Increase offsets retention default to 7 days2.0.0
9190KIP-183 - Change PreferredReplicaLeaderElectionCommand to use AdminClient2.0.0
9291KIP-182: Reduce Streams DSL overloads and allow easier use of custom storage engines1.0.0
9392KIP-180: Add a broker metric specifying the number of consumer group rebalances in progress1.1.0
9493KIP-177: Consumer perf tool should count rebalance time1.0.0
9594KIP-176: Remove deprecated new-consumer option for tools2.0.0
9695KIP-175: Additional '--describe' views for ConsumerGroupCommand1.1.0
9796KIP-174 - Deprecate and remove internal converter configs in WorkerConfig2.0.0
9897KIP-173: Add prefix to StreamsConfig to enable setting default internal topic configs1.0.0
9998KIP-171 - Extend Consumer Group Reset Offset for Stream Application1.1.0
10099KIP-168: Add GlobalTopicCount and GlobalPartitionCount metric per cluster1.0.0
101100KIP-167: Add interface for the state store restoration process1.0.0
102101

KIP-164 - Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics

1.0.0
103102

KIP-163: Lower the Minimum Required ACL Permission of OffsetFetch

1.0.0
104103KIP-162: Enable topic deletion by default1.0.0
105104KIP-161: streams deserialization exception handlers1.0.0
106105KIP-160: Augment KStream.print(), KStream.writeAsText() to allow users pass in extra parameters in the printed string1.0.0
107106KIP-157 - Add consumer config options to streams reset tool1.0.0
108107KIP-156 Add option "dry run" to Streams application reset tool0.11.0.0
109108KIP-155 - Add range scan for windowed state stores0.11.0.0
110109KIP-154 Add Kafka Connect configuration properties for creating internal topics0.11.0.0
111110KIP-153: Include only client traffic in BytesOutPerSec metric0.11.0.0
112111KIP-152 - Improve diagnostics for SASL authentication failures1.0.0
113112KIP-151 Expose Connector type in REST API0.11.0.0
114113KIP-150 - Kafka-Streams Cogroup2.1.0 (WIP)
115114KIP-149: Enabling key access in ValueTransformer, ValueMapper, and ValueJoiner1.1.0 (partially implemented)
(WIP for 2.1.0)
116115KIP-146 - Classloading Isolation in Connect

0.11.0.0

117116KIP-145 - Expose Record Headers in Kafka Connect1.1.0
118117KIP-144: Exponential backoff for broker reconnect attempts0.11.0.0
119118KIP-143: Controller Health Metrics0.11.0.0
120119KIP-140: Add administrative RPCs for adding, deleting, and listing ACLs0.11.0.0
121120KIP-138: Change punctuate semantics1.0.0
122121KIP-137: Enhance TopicCommand --describe to show topics marked for deletion0.11.0.0
123122KIP-136: Add Listener name to SelectorMetrics tags0.11.0.0
124123KIP-134: Delay initial consumer group rebalance0.11.0.0
125124KIP-133: Describe and Alter Configs Admin APIs0.11.0.0
126125KIP-130: Expose states of active tasks to KafkaStreams public API1.0.0
127126KIP-129: Streams Exactly-Once Semantics0.11.0.0
128127KIP-128: Add ByteArrayConverter for Kafka Connect0.11.0.0
129128KIP-126 - Allow KafkaProducer to split and resend oversized batches.0.11.0.0
130129KIP-124 - Request rate quotas0.11.0.0
131130KIP-123: Allow per stream/table timestamp extractor0.11.0.0
132131KIP-122: Add Reset Consumer Group Offsets tooling0.11.0.0
133132KIP-121: Add KStream peek method

0.11.0.0

134133KIP-120: Cleanup Kafka Streams builder API1.0.0
135134KIP-119: Drop Support for Scala 2.10 in Kafka 0.110.11.0.0
136135KIP-118: Drop Support for Java 72.0.0
137136KIP-117: Add a public AdminClient API for Kafka admin operations0.11.0.0
138137KIP-115: Enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation0.11.0.0
139138KIP-114: KTable state stores and improved semantics0.11.0.0
140139KIP-113: Support replicas movement between log directories1.1.0
141140KIP-112: Handle disk failure for JBOD1.0.0
142141KIP-109: Old Consumer Deprecation0.11.0.0
143142KIP-108: Create Topic Policy0.10.2.0
144143KIP-107: Add deleteRecordsBefore() API in AdminClient0.11.0.0
145144

KIP-106 - Change Default unclean.leader.election.enabled from True to False

0.11.0.0
146145KIP-105: Addition of Record Level for Sensors0.10.2.0
147146KIP-104: Granular Sensors for Streams 0.10.2.0
148147KIP-103: Separation of Internal and External traffic0.10.2.0
149148KIP-102 - Add close with timeout for consumers0.10.2.0
150149KIP-101 - Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation0.11.0.0
151150KIP-100 - Relax Type constraints in Kafka Streams API0.10.2.0
152151KIP-99: Add Global Tables to Kafka Streams0.10.2.0
153152KIP-98 - Exactly Once Delivery and Transactional Messaging0.11.0.0
154153KIP-97: Improved Kafka Client RPC Compatibility Policy0.10.2.0
155154KIP-96 - Add per partition metrics for in-sync and assigned replica count0.10.2.0
156155KIP-94 Session Windows0.10.2.0
157156KIP-93: Improve invalid timestamp handling in Kafka Streams0.10.2.0
158157KIP-92 - Add per partition lag metrics to KafkaConsumer0.10.2.0
159158KIP-91 Provide Intuitive User Timeouts in The Producer2.1.0
160159KIP-90 - Remove zkClient dependency from Streams0.10.2.0
161160KIP-89: Allow sink connectors to decouple flush and offset commit0.10.2.0
162161KIP-88: OffsetFetch Protocol Update0.10.2.0
163162KIP-86: Configurable SASL callback handlers2.0.0
164163KIP-85: Dynamic JAAS configuration for Kafka clients0.10.2.0
165164KIP-84: Support SASL SCRAM mechanisms0.10.2.0
166165KIP-82 - Add Record Headers0.11.0.0
167166KIP-81: Bound Fetch memory usage in the consumer2.0.0 (WIP)
168167KIP-79 - ListOffsetRequest/ListOffsetResponse v1 and add timestamp search methods to the new consumer0.10.1.0
169168KIP-78: Cluster Id0.10.1.0
170169KIP-77: Improve Kafka Streams Join Semantics0.10.2.0
171170KIP-75 - Add per-connector Converters0.10.1.0
172171KIP-74: Add Fetch Response Size Limit in Bytes0.10.1.0
173172KIP-73: Replication Quotas0.10.1.0
174173KIP-72: Allow putting a bound on memory consumed by Incoming request 1.0.0
175174KIP-71: Enable log compaction and deletion to co-exist0.10.1.0
176175KIP-70: Revise Partition Assignment Semantics on New Consumer's Subscription Change0.10.1.0
177176KIP-67: Queryable state for Kafka Streams0.10.1.0
178177KIP-66: Single Message Transforms for Kafka Connect0.10.2.0 / 0.11.0.0
179178KIP-65: Expose timestamps to Connect0.10.1.0
180179KIP-63: Unify store and downstream caching in streams0.10.1.0
181180KIP-62: Allow consumer to send heartbeats from a background thread0.10.1.0
182181KIP-60 - Make Java client classloading more flexible0.10.1.0
183182KIP-58 - Make Log Compaction Point Configurable0.10.1.0
184183KIP-57 - Interoperable LZ4 Framing0.10.0.0
185184KIP-56: Allow cross origin HTTP requests on all HTTP methods0.10.0.0
186185KIP-55: Secure Quotas for Authenticated Users0.10.1.0
187186KIP-54: Sticky Partition Assignment Strategy0.11.0.0
188187KIP-52: Connector Control APIs0.10.0.0
189188KIP-51 - List Connectors REST API0.10.0.0
190189KIP-50 - Move Authorizer to o.a.k.common package0.10.1.0
191190KIP-48 Delegation token support for Kafka1.1.0
192191KIP-45 - Standardize all client sequence interaction on j.u.Collection.0.10.0.0
193192KIP-43: Kafka SASL enhancements0.10.0.0
194193KIP-42: Add Producer and Consumer Interceptors0.10.0.0
195194KIP-41: Consumer Max Records0.10.0.0
196195KIP-40: ListGroups and DescribeGroup0.9.0.0
197196KIP-38: ZooKeeper Authentication0.9.0.0
198197KIP-36 - Rack aware replica assignment0.10.0.0
199198KIP-35 - Retrieving protocol version0.10.0.0
200199KIP-33 - Add a time based log index0.10.1.0
201200KIP-32 - Add timestamps to Kafka message0.10.0.0
202201KIP-31 - Move to relative offsets in compressed message sets0.10.0.0
203202KIP-28 - Add a processor client0.10.0.0
204203KIP-26 - Add Kafka Connect framework for data import/export0.9.0.0
205204KIP-25 - System test improvements0.9.0.0
206205KIP-22 - Expose a Partitioner interface in the new producer0.9.0.0
207206KIP-21 - Dynamic Configuration0.9.0.0 (WIP)
208207KIP-20 Enable log preallocate to improve consume performance under windows and some old Linux file system0.9.0.0
209208KIP-19 - Add a request timeout to NetworkClient0.9.0.0
210209KIP-16 - Automated Replica Lag Tuning0.9.0.0
211210KIP-15 - Add a close method with a timeout in the producer0.9.0.0
212211KIP-13 - Quota Design0.9.0.0
213212KIP-12 - Kafka Sasl/Kerberos and SSL implementation0.9.0.0
214213KIP-11 - Kafka Authorizer design0.9.0.0
215214KIP-8 - Add a flush method to the producer API0.9.0.0
216215KIP-4 - Metadata Protocol Changes0.10.0.0
217216KIP-4 - Command line and centralized administrative operations0.9.0.0, 0.10.0.0, 0.10.1.0
218217KIP-3 - Mirror Maker Enhancement0.9.0.0
219218KIP-2 - Refactor brokers to allow listening on multiple ports and IPs0.9.0.0
220219KIP-1 - Remove support of request.required.acks0.9.0.0

...

KIPComment
KIP-59: Proposal for a kafka broker commandSent emails to Dev discussion group. Work tracked under KAFKA-3663.
KIP-110: Add Codec for ZStandard Compression 
KIP-125: ZookeeperConsumerConnector to KafkaConsumer Migration and Rollback 
KIP-131 - Add access to OffsetStorageReader from SourceConnector 
KIP-135 : Send of null key to a compacted topic should throw non-retriable error back to user 
KIP 141 - ProducerRecord & SourceRecord: Add timestamp constructors 
KIP-142: Add ListTopicsRequest to efficiently list all the topics in a cluster 
KIP-148: Add a connect timeout for client 
KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics 
KIP-159: Introducing Rich functions to Streams 
KIP-165: Extend Interactive Queries for return latest update timestamp per key

 

KIP-166 - Add a tool to make amounts of replicas and leaders on brokers balanced 
KIP-169 - Lag-Aware Partition Assignment Strategy 
KIP-178: Size-based log directory selection strategy

 

KIP-185: Make exactly once in order delivery the default producer setting

 
KIP-193: Add SchemaBuilder.from(Schema) 
KIP-199: Add Kafka Connect offset tool 
KIP-201: Rationalising Policy interfaces 
KIP-209: Connection String Support 
KIP-213 Support non-key joining in KTable 
KIP-216: IQ should throw different exceptions for different errors 
KIP-217: Expose a timeout to allow an expired ZK session to be re-created 
KIP-221: Repartition Topic Hints in Streams 
KIP-228 Negative record timestamp supportVoting in progress
KIP 230: Name Windowing JoinsDiscussion
KIP-232: Detect outdated metadata using per-partition leaderEpoch fieldVoting in progress
KIP-234: add support for getting topic defaults from AdminClient 
KIP-235: Add DNS alias support for secured connectionAccepted
KIP-236: Interruptible Partition Reassignment 
KIP-240: AdminClient.listReassignments() AdminClient.describeReassignments() 
KIP-242: Mask password in Kafka Connect Rest API response 
KIP-248 - Create New ConfigCommand That Uses The New AdminClientVoting in progress (pending on more binding votes)
KIP-250 Add Support for Quorum-based Producer Acknowledgment 
KIP-252 - Extend ACLs to allow filtering based on ip ranges and subnets 
KIP-253: Support in-order message delivery with partition expansion 
KIP-254: JsonConverter Exception Handeling 
KIP-258: Allow to Store Record Timestamps in RocksDBdepends on KIP-268
KIP-259: Improve Streams DSL Timestamp Propagation Semantics 
KIP-260: add primary join operation for Stream-Stream join (WIP)Draft
KIP-262: Metadata should include number of state stores for taskdepends on KIP-268
KIP-264: Add a consumer metric to record raw fetch sizeVoting in progress
KIP-271: Add NetworkClient redirectorDiscussion
KIP-273: Kafka to support using ETCD beside ZookeeperDiscussion
KIP-275 - Indicate "isClosing" in the SinkTaskContextVoting in progress
KIP-280: Enhanced log compactionDiscussion

KIP-291: Separating controller connections and requests from the data plane

Voting in progress
KIP-293 Add new metrics for consumer/replication fetch requestsVoting in progress
KIP-296: Add connector level configurability for producer/consumer client configs Discussion
KIP-300: Add Windowed KTable API in StreamsBuilderDiscussion
KIP-301: Schema Inferencing for JsonConverterDiscussion
KIP-302 - Enable Kafka clients to use all DNS resolved IP addressesDiscussion
KIP-304: Connect runtime mode improvements for container platformsDiscussion
KIP-312 Add Overloaded StreamsBuilder Build Method to Accept java.util.PropertiesAccepted
KIP-313: Add KStream.flatTransform and KStream.flatTransformValues Accepted
KIP-314: KTable to GlobalKTable Bi-directional JoinDiscussion
KIP-315: Stream Join Sticky AssignorDiscussion
KIP-316: Command-line overrides for ConnectDistributed worker propertiesDiscussion
KIP-317: Add transparent data encryption functionalityDiscussion
KIP-320: Allow fetchers to detect and handle log truncationDiscussion
KIP-325: Extend Consumer Group Command to Show Beginning OffsetsVoting in Progress
KIP-326: Schedulable KTable as Graph sourceDiscussion
KIP-331 Add default implementation to close() and configure() for Serializer, Deserializer and Serde
KIP-333: Add faster mode of rebalancingDiscussion
KIP-334 - Include partitions in exceptions raised during consumer record deserialization/validation
KIP-335: Consider configurations for KafkaStreams

Discussion

KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/DeserializerDiscussion
KIP-339: Create a new IncrementalAlterConfigs API
KIP-345: Reduce multiple consumer rebalances by specifying member id
KIP-346 - Improve LogCleaner behavior on error
KIP-347: Enable batching in FindCoordinatorRequestDiscussion
KIP-348 Eliminate null from SourceTask#poll()
KIP-350: Allow kafka-topics.sh to take brokerid as parameter to show partitions associated with it
KIP-351: Add --critical-partitions option to describe topics commandDiscussion
KIP-354: Add a Maximum Log Compaction LagDiscussion
KIP-356: Add KafkaConsumer fetch-error-rate and fetch-error-total metricsDiscussion
KIP-357: Add support to list ACLs per principalDiscussion
KIP-358: Migrate Streams API to Duration instead of long ms timesDiscussion
KIP-360: Improve handling of unknown producerDiscussion
KIP-361: Add Consumer Configuration to Disable Auto Topic CreationDiscussion
KIP-362: Support Dynamic Session WindowDiscussion
KIP-363: Allow performance tools to print final results to output fileDiscussion
KIP-364: Remove implicit Materialized, Consumed and ProducedDiscussion
KIP-365: Materialized, Serialized, Joined, Consumed and Produced with implicit SerdeDiscussion
KIP-366: Make FunctionConversions privateVoting
KIP-367 Introduce close(Duration) to Producer and AdminClient instead of close(long, TimeUnit)
KIP 368: Allow SASL Connections to Periodically Re-AuthenticateDiscussion
KIP-370: Remove Orphan Partitions

...