Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

KIP round-up

Next KIP Number: 10181020

Use this number as the identifier for your KIP and increment this value.

...

KIPComment
KIP-59: Proposal for a kafka broker commandSent emails to Dev discussion group. Work tracked under KAFKA-3663.
KIP-125: ZookeeperConsumerConnector to KafkaConsumer Migration and Rollback
KIP-135 : Send of null key to a compacted topic should throw non-retriable error back to user
KIP 141 - ProducerRecord & SourceRecord: Add timestamp constructors
KIP-142: Add ListTopicsRequest to efficiently list all the topics in a cluster
KIP-148: Add a connect timeout for client
KIP-166 - Add a tool to make amounts of replicas and leaders on brokers balanced
KIP-169 - Lag-Aware Partition Assignment Strategy
KIP-178: Size-based log directory selection strategy


KIP-185: Make exactly once in order delivery the default producer setting


KIP-193: Add SchemaBuilder.from(Schema)
KIP-201: Rationalising Policy interfaces

KIP-209 - Connection String Support


KIP-217: Expose a timeout to allow an expired ZK session to be re-created
KIP-228 Negative record timestamp supportVoting in progress
KIP-234: add support for getting topic defaults from AdminClient
KIP-236: Interruptible Partition Reassignment Discussion
KIP-240: AdminClient.listReassignments() AdminClient.describeReassignments()
KIP-242: Mask password in Kafka Connect Rest API response
KIP-250 Add Support for Quorum-based Producer Acknowledgment
KIP-252 - Extend ACLs to allow filtering based on ip ranges and subnets
KIP-253: Support in-order message delivery with partition expansion Discussion
KIP-254: JsonConverter Exception Handeling
KIP-264: Add a consumer metric to record raw fetch sizeVoting in progress
KIP-271: Add NetworkClient redirectorDiscussion
KIP-273: Kafka to support using ETCD beside ZookeeperDiscussion
KIP-293 Add new metrics for consumer/replication fetch requestsVoting in progress
KIP-296: Add connector level configurability for producer/consumer client configs Discussion
KIP-301: Schema Inferencing for JsonConverterDiscussion
KIP-304: Connect runtime mode improvements for container platformsDiscussion
KIP-315: Stream Join Sticky AssignorDiscussion
KIP-316: Command-line overrides for ConnectDistributed worker propertiesDiscussion
KIP-317: Add end-to-end data encryption functionality to Apache KafkaDiscussion
KIP-325: Extend Consumer Group Command to Show Beginning OffsetsVoting in Progress345
KIP-333: Add faster mode of rebalancingDiscussion
KIP-334 - Include partitions in exceptions raised during consumer record deserialization/validation
KIP-348 Eliminate null from SourceTask#poll()
KIP-350: Allow kafka-topics.sh to take brokerid as parameter to show partitions associated with it
KIP-356: Add KafkaConsumer fetch-error-rate and fetch-error-total metricsDiscussion
KIP-363: Allow performance tools to print final results to output fileDiscussion
KIP-370: Remove Orphan PartitionsDiscussion
KIP-375: Kafka Clients - make Metadata#TOPIC_EXPIRY_MS configurableDiscussion
KIP-381: Connect: Tell about records that had their offsets flushed in callback

Voting in progress (restarted 18th January 2019, due to no votes in first attempt)

KIP-384: Add config for incompatible changes to persistent metadataDiscussion
KIP-385: Avoid throwing away prefetched dataDiscussion
KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

Discussion

KIP-388: Add observer interface to record request and responseDiscussion
KIP-395: Encypt-then-MAC Delegation token metadata
KIP-400: Improve exit status in case of errors in ConsoleProducerDiscussion
KIP-403: Increase ProducerPerformance precision by using nanoTimeDraft
KIP-406: GlobalStreamThread should honor custom offset policy.Blocked
KIP-407: Kafka Connect support override worker kafka api configuration with connector configuration that post by rest api

KIP-409: Allow creating under-replicated topics and partitions


KIP-410: Add metric for request handler thread pool utilization by request type

Discussion
KIP-416: Notify SourceTask of ACK'd offsets, metadataDiscussion
KIP-419: Safely notify Kafka Connect SourceTask is stoppedVoting in progress, JIRA exists with pull request
KIP-423: Add JoinReason to Consumer Join Group Protocol

Discussion

KIP-424: Allow suppression of intermediate events based on wall clock timeDiscussion
KIP-426: Persist Broker Id to ZookeeperDiscussion
KIP-435: Internal Partition Reassignment BatchingDiscussion
KIP-438: Expose task, connector IDs in Connect APIDiscussion
KIP-439: Cleanup built-in Store interfacesDiscussion
KIP-448: Add State Stores Unit Test Support to Kafka Streams Test UtilsDiscussion
KIP-452: Tool to view cluster statusDiscussion, JIRA exists
KIP-468: Avoid decompression of record when validate record at server in the scene of inPlaceAssignment .Under discussion
KIP-473: Enable KafkaLog4JAppender to use SASL Authentication Callback HandlersUnder Discussion 
KIP-477: Add PATCH method for connector config in Connect REST APIUnder Discussion
KIP-487: Automatic Topic Creation on ProducerUnder Discussion
KIP-489: Kafka Consumer Record Latency MetricUnder Discussion
KIP-490: New metric to count offsets expired without being consumed by a consumer groupUnder discussion
KIP-491: Preferred Leader Deprioritized List (Temporary Blacklist)Under discussion
KIP-494: Connect REST Endpoint to get Transformations (SMTs)
KIP-498: Add client-side configuration for maximum response size to protect against OOMSent emails to Dev discussion group. Work tracked under KAFKA-4090.
KIP-502: Connect SinkTask.put(...) to specify ArrayList<SinkRecord> in Signature
KIP-505: Add new public method to only update assignment metadata in consumerUnder discussion
KIP-506: Allow setting SCRAM password via Admin interfaceUnder discussion
KIP-508: Make Suppression State QueriableUnder discussion
KIP-509: Rebalance and restart ProducersUnder discussion
KIP-510: Metrics library upgradeUnder discussion
KIP-512: Adding headers to RecordMetaDataSent mail to dev discussion group
KIP-513: Distinguish between Key and Value serdes in scala wrapper library for kafka streamsSent mail to dev discussion group
KIP-536: Propagate broker start time to Admin APIUnder discussion
KIP-539: Add mechanism to flush records out in low volume suppression buffersUnder discussion.
KIP-540: Implement per key stream time trackingDraft (In Progress)
KIP-542: Partition Reassignment ThrottlingDraft
KIP-547: Extend ConsumerInterceptor to allow modification of Consumer CommitsUnder Discussion
KIP-548 Add Option to enforce rack-aware custom partition reassignment executionUnder Discussion
KIP-549: Surface Consumer's Metadata in KafkaAdminClient#describeConsumerGroupsUnder Discussion
KIP-550: Mechanism to Delete Stray Partitions on BrokerDraft
KIP-552: Suppress "unused config" warning

Under Discussion

KIP-560: Auto infer external topic partitions in stream reset tool

Under Discussion

KIP-561: Regex Support for ConsumerGroupCommand

Under Discussion
KIP-563: Add 'tail -n' feature for ConsoleConsumerUnder Discussion
KIP-564: Add new cached authorizer:change the dim of cacheUnder Discussion
KIP-565: Using AclCommand,avoid call the global method loadcache in SimpleAclAuthorizerUnder Discussion
KIP-566: Add rebalance callbacks to ConsumerInterceptorUnder Discussion
KIP-567: Kafka Cluster Audit

Under Discussion

KIP-575: build a Kafka-Exporter by JavaUnder Discussion
KIP-576: Support dynamic update of more broker configs related to replicationUnder Discussion
KIP-578: Add configuration to limit number of partitionsVoting
KIP-579: new exception on min.insync.replicas > replication.factorVoting in progress
KIP-582: Add a "continue" option for Kafka Connect error handlingUnder Discussion
KIP-583: add tag "partition" to BrokerTopicMetrics so as to observe the partition metrics on the same brokerUnder Discussion
KIP-587: Suppress detailed responses for handled exceptions in security-sensitive environmentsUnder Discussion
KIP-592: Replicate mirrormaker topics from earliestUnder Discussion
KIP-595: A Raft Protocol for the Metadata QuorumUnder Discussion
KIP-596: Safely abort Producer transactions during application shutdownUnder Discussion
KIP-598: Augment TopologyDescription with store and source / sink serde informationUnder Discussion
KIP-619: Add internal topic creation supportUnder Discussion
KIP-625: Richer encodings for integral-typed protocol fieldsUnder Discussion

KIP-628: ConsumerPerformance's multi-thread implementation

Sent email to Dev discussion group. Work tracked under KAFKA-10136.
KIP-636: Make RPC error codes and messages tagged fieldsUnder Discussion
KIP-637: Include min.insync.replicas in MetadataResponse to make Producer smarter in partitioning eventsDraft
KIP-639: Move nodeLevelSensor and storeLevelSensor methods from StreamsMetricsImpl to StreamsMetricsUnder Discussion
KIP-640: Add log compression analysis toolUnder Discussion
KIP-649: Dynamic client configurationUnder Discussion
KIP-655: Windowed Distinct Operation for Kafka Streams APIVoting in progress
KIP-656: MirrorMaker2 Exactly-once SemanticsDraft
KIP-665: Kafka Connect Hash SMT

Voting in progress

KIP-668: Expose REST endpoint to list converter pluginsUnder Discussion
KIP-669 Preserve Source Partition in Kafka Streams from contextDraft

KIP-674: API to Aggregate Metrics in Kafka Streams

Under Discussion

KIP-675: Convert a KTable to a KStream using the previous valueUnder Discussion
KIP-678: New Kafka Connect SMT for plainText => struct with RegexVoting
KIP-681: Rename master key in delegation token featureDraft
KIP-682: Connect TimestampConverter support for multiple fields and multiple input formatsUnder Discussion
KIP-683: Add recursive support to Connect Cast and ReplaceField transforms, and support for casting complex types to either a native or JSON stringUnder Discussion
KIP-685: Loosen permission for listing reassignmentsUnder Discussion
KIP-686: API to ensure Records policy on the brokerUnder Discussion
KIP-687: Automatic Reloading of Security StoreUnder Discussion
KIP-688: Support dynamic update of delete.topic.enable config

Under Discussion

KIP-692: Make AdminClient value object constructors publicUnder discussion

KIP-693: Client-side Circuit Breaker for Partition Write Errors

Voting in progress

KIP-694: Support Reducing Partitions for Topics

Under Discussion
KIP-697: Stricter parsing of addresses in configsUnder Discussion
KIP-701: Add --override option to bin/connect-distributed.shUnder Discussion
KIP-702: The control plane needs to force the validation of requests from the controllerUnder Discussion
KIP-703: Add a metric for reporting idle connections closed

Under Discussion

KIP-705: Selectively Disable Topology OptimizationsDraft
KIP-706: Add method "Producer#produce" to return CompletionStage instead of FutureUnder Discussion
KIP-711 Deprecate org.apache.kafka.streams.errors.BrokerNotFoundExceptionUnder Discussion
KIP-712: Shallow MirroringUnder Discussion
KIP-713: Validation of Enums in configurationUnder Discussion
KIP-718: Make KTable Join on Foreign key unopinionatedUnder Discussion
KIP-723: Add socket.tcp.no.delay property to Kafka ConfigUnder Discussion

KIP-727: Add --under-preferred-replica-partitions option to describe topics command

Under Discussion

KIP-729: Custom validation of records on the broker prior to log append

Under Discussion
KIP-731: Record Rate Limiting for Kafka ConnectUnder Discussion
KIP-736: Report the true end to end fetch latencyUnder Discussion
KIP-737: Add canTrackSource to ReplicationPolicyUnder Discussion
KIP-739: Block Less on KafkaProducer#sendUnder Discussion
KIP-749: Add --files and --file-separator options to the ConsoleProducerUnder Discussion
KIP-752: Support --bootstrap-server in ReplicaVerificationToolUnder Discussion
KIP-754: Make Scala case class's finalUnder Discussion
KIP-755: Add new AUTO_CREATE ACL for auto topic creationUnder Discussion
KIP-759: Unneeded repartition cancelingAccepted, targeting 3.7
KIP-760: Minimum value for segment.ms and segment.bytesUnder Discussion
KIP-762: Delete Committed Connect RecordsUnder Discussion
KIP-767: Connect Latency MetricsUnder Discussion
KIP-774: Deprecate public access to Admin client's *Result constructors

Under Discussion

KIP-777: Improved testability for Admin clientDraft
KIP-780: Support fine-grained compression optionsUnder Discussion
KIP-781: Improve MirrorMaker2's client configurationUnder Discussion
KIP-782: Expandable batch size in producerVoting
KIP-785: Automatic storage formattingUnder Discussion
KIP-786: Emit Metric Client Quota ValuesUnder Discussion 
KIP-789: Use localhost:9092 as default bootstrap-server/broker-list in client toolsKIP page WIP
KIP-799: Align behaviour for producer callbacks with documented behaviourUnder Discussion

KIP-802: Validation Support for Kafka Connect SMT Options

Under Discussion

KIP-803: Add Task ID and Connector Name to Connect Task ContextUnder Discussion
KIP-804: OfflinePartitionsCount Tagged by TopicUnder Discussion
KIP-807: Refactor KafkaStreams exposed metadata hierarchyUnder Discussion
KIP-809: Support live upgrades with dynamic addition/removal of modular topologiesKIP page WIP
KIP-816: Topology changes without local state resetUnder Discussion
KIP-817: Fix inconsistency in dynamic application log levelsUnder Discussion
KIP-819: Merge multiple KStreams in one operationUnder Discussion
KIP-822: Optimize the semantics of KafkaConsumer#pause to be consistent between the two RebalanceProtocolsUnder Discussion
KIP-823: Update Admin::describeConfigs to allow fetching specific configurationsKIP page WIP

KIP-826: Define platforms supported

Under Discussion
KIP-828: Add the corresponding validator to the configuration where the validator is missingUnder Discussion
KIP-829: (console-consumer) add print.topic propertyUnder Discussion
KIP-838 Simulate batching and compressionUnder Discussion

KIP-839: Provide builders for KafkaProducer/KafkaConsumer and KafkaStreams

Under Discussion
KIP-842: Add richer group offset reset mechanismsUnder Discussion
KIP-849: Expose logdirs total and usable space via kafka-log-dirs.shUnder Discussion
KIP-853: KRaft Voter ChangesUnder Discussion
KIP-855: Add schema.namespace parameter to SetSchemaMetadata SMT in Kafka ConnectUnder Discussion
KIP-856: KRaft Disk Failure RecoveryDraft
KIP-857: Streaming recursion in Kafka StreamsUnder Discussion

KIP-860: Add client-provided option to guard against replication factor change during partition reassignments

Under Discussion
KIP-864: Add End-To-End Latency Metrics to ConnectorsUnder Discussion
KIP-870: Retention policy based on record event timeUnder Discussion
KIP-873: ExceptionHandlingDeserializer, RetryDeserializer, PipeSerializer, PipeDeserializerUnder Discussion

KIP-877: Mechanism for plugins and connectors to register metrics

Under Discussion
KIP-879: Multi-level Rack AwarenessUnder Discussion
KIP-880: X509 SAN based SPIFFE URI ACL within mTLS Client CertificatesUnder Discussion
KIP-882: Kafka Connect REST API configuration validation timeout improvementsUnder Discussion

KIP-883: Add isDeleted flag when stopping a connector

Under Discussion
KIP-885: Expose Broker's Name and Version to ClientsUnder Discussion
KIP-886: Add Client Producer and Consumer BuildersUnder Discussion

KIP-888: Batch describe ACLs and describe client quotas

Under Discussion

KIP-891: Running multiple versions of a connectorUnder Discussion
KIP-895: Dynamically refresh partition count of __consumer_offsetsUnder Discussion
KIP-899: Allow clients to rebootstrap

Under Discussion

KIP-901: Add connectorDeleted flag when stopping tasks

Under Discussion

KIP-905: Broker interceptors

Under Discussion

KIP-910: Update Source offsets for Source Connectors without producing records

Under Discussion

KIP-912: Support decreasing send's block time without worrying about metadata's fetch 

Under Discussion

KIP-913: add new method to provide possibility for accelerate first record's sending

Under Discussion

KIP-916: MM2 distributed mode flow log context

Under Discussion
KIP-917: Additional custom metadata for remote log segmentUnder Discussion

KIP-918: MM2 Topic And Group Listener

Under Discussion

KIP-921: OpenJDK CRaC support

Under Discussion

KIP-922: Add the traffic metric of the partition dimension

Under Discussion

KIP-924: Introduce public config for Streams task assignor

Under Discussion

KIP-926: introducing acks=min.insync.replicas config

Under Discussion

KIP-928: Making Kafka resilient to log directories becoming full

Under Discussion

KIP-930: Rename ambiguous Tiered Storage Metrics

Under Discussion

KIP-931: Flag to ignore unused message attribute field

Under Discussion

KIP-932: Queues for Kafka

Under Discussion

KIP-933: Publish metrics when source connector fails to poll data

Under Discussion

KIP-934: Add DeleteTopicPolicy

Under Discussion

KIP-935: Extend AlterConfigPolicy with existing configurations

Under Discussion

KIP-936: Throttle number of active PIDs

Under Discussion

KIP-939: Support Participation in 2PC

Under Discussion

KIP-940: Broker extension point for validating record contents at produce time

Under Discussion

KIP-943: Add independent “offset.storage.segment.bytes” for connect-distributed.properties

Under Discussion

KIP-945: Update threading model for ConsumerWIP
KIP-946: Modify exceptions thrown by Consumer APIsWIP

KIP-948: Allow custom prefix for internal topic names in Kafka Streams

Under Discussion

KIP-950: Tiered Storage Disablement

Under Discussion

KIP-952: Regenerate segment-aligned producer snapshots when upgrading to a Kafka version supporting Tiered Storage

Under Discussion

KIP-953: partition method to be overloaded to accept headers as well.

Draft

KIP-955: Add stream-table join on foreign key

Under Discussion

KIP-956 Tiered Storage Quotas

Draft

KIP-962: Relax non-null key requirement in Kafka Streams

Under Discussion

KIP-964 Have visibility when produce requests become "async"   

Under discussion

KIP-965: Support disaster recovery between clusters by MirrorMaker

Under Discussion

KIP-969: Support range Interactive Queries (IQv2) for Versioned State Stores 


KIP-971: Expose replication-record-lag MirrorMaker2 metric

KIP-972: Add the metric of the current running version of kafka

Under Discussion

KIP-973: Expose per topic replication rate metrics

Under Discussion

KIP-974: Docker Image for GraalVM based Native Kafka Broker

Under Discussion

KIP-981: Manage Connect topics with custom implementation of Admin

Under Discussion

KIP-982: Access SslPrincipalMapper and kerberosShortNamer in Custom KafkaPrincipalBuilder

Voting in progress

(PR is reviewed by Contributors)

KIP-983: Full speed async processing during rebalance

Draft

KIP-984: Add pluggable compression interface to Kafka

Under Discussion

KIP-986: Cross-Cluster Replication

Draft

KIP-987: Connect Static Assignments

Under Discussion

KIP-989: RocksDB Iterator Metrics

Under Discussion

KIP-990: Capability to PAUSE Tasks on DeserializationException

Under Discussion

KIP-991: Allow DropHeaders SMT to drop headers by wildcard/regexp

Under Discussion

KIP-995: Allow users to specify initial offsets while creating connectors

Under Discussion

KIP-997: update WindowRangeQuery and unify WindowKeyQuery and WindowRangeQuery

Under Discussion 
Vote

KIP-999: Server-side Consumer Lag Metrics

Draft

KIP-1002: Fetch remote segment indexes at once

Under Discussion

KIP-1003: Signal next segment when remote fetching

Under Discussion

KIP-1006: Remove SecurityManager Support

Draft

KIP-1008: ParKa - the Marriage of Parquet and Kafka

Under Discussion

KIP-1009: Add Broker-level Throttle Configurations

Under Discussion

KIP-1010: Topic Partition Quota

Under Discussion

KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

Under Discussion

KIP-1015: Limit number of ssl connections in brokers

Under Discussion

KIP-1016 Make MM2 heartbeats topic name configurable

Under Discussion

KIP-1017: Health check endpoint for Kafka Connect

Under Discussion

KIP-1018: Introduce max remote fetch timeout config for DelayedRemoteFetch requests

Under Discussion

KIP-1019: Expose method to determine Metric Measurability

Under Discussion


Go to link

...