This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state:   [One of "Under Discussion", "Accepted", "Rejected"]

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: here [Change the link from KAFKA-1 to your own ticket] 

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

...

Currently, the ProducerPerformance tool supports transactional producers but cannot randomly abort transactions during testing. This prevents developers from evaluating how transactional producers behave under failure scenarios.

This proposal targets four critical areas:

1. Measuring the overhead costs of aborting transactions;
2. Assessing how failed transactions affect producer throughput;
3. Validating transaction recovery processes;
4. Benchmarking transactional producers under realistic fault conditions.

Public Interfaces

Briefly list any new interfaces that will be introduced as part of this proposal or any existing interfaces that will be removed or changed. The purpose of this section is to concisely call out the public contract that will come along with this feature.

A public interface is any change to the following:

• Binary log format

• The network protocol and api behavior

• Any class in the public packages under clientsConfiguration, especially client configuration

  • org/apache/kafka/common/serialization

  • org/apache/kafka/common

  • org/apache/kafka/common/errors

  • org/apache/kafka/clients/producer

  • org/apache/kafka/clients/consumer (eventually, once stable)

• Monitoring

• Command line tools and arguments

• Anything else that will likely break existing users in some way when they upgrade

Proposed Changes

A single new command-line option is introduced: --transaction-abort-ratio

Property

Description

Type

Double 

Range

0.0 to 1.0 

Default

0.0 (no transactions aborted)

Dependency

Only valid when transactions are enabled

• At `1.0`: all transactions are aborted;
• At `0.5`: approximately half of the transactions are aborted;
• At `0.0` (default): behavior is identical to the current version.

Proposed Changes

1. Argument parsing

Add the --transaction-abort-ratio  argument after the existing --transaction-duration-ms argument:

parser.addArgument("--transaction-abort-ratio")
        .action(store())
        .required(false)
        .type(Double.class)
        .metavar("TRANSACTION-ABORT-RATIO")
        .dest("transactionAbortRatio")
        .setDefault(0.0)
        .help("The ratio of transactions to abort during the test. "
                + "The value should be between 0.0 and 1.0. "
                + "This option is only valid when transactions are enabled.");

2. Configuration validation

Add a new field double transactionAbortRatio  with the following validations:

• Throw ArgumentParserException if the value is outside the [0.0, 1.0] range;
• Throw ArgumentParserException if transactionAbortRatio is greater than 0.0 but transactions are not enabled, since aborting without transactions is meaningless.

3. Abort logic

There are currently two places where commitTransaction() is called:

• In-loop: when the transaction duration exceeds transactionDurationMs
• Post-loop: to handle remaining uncommitted records

At both commit points, use the existing SplittableRandom instance to randomly decide whether to commit or abort based on the ratio:

if (random.nextDouble() < config.transactionAbortRatio) {
    producer.abortTransaction();
} else {
    producer.commitTransaction();
}

Note: The current code uses a fixed seed (new SplittableRandom(0)). The abort decision reuses the same instance, so for a given ratio and record count, results are deterministic and reproducible.

4. Warmup phase behavior

Transactions during the warmup phase are also subject to the --transaction-abort-ratio. The warmup phase (introduced in KAFKA-17645) is designed to bring the system into a steady state before collecting performance statistics. If the abort ratio were only applied during the steady-state phase, the system would transition from an all-commit warmup to a mixed commit/abort steady state, introducing an additional settling period that undermines the purpose of warmup. Applying the same abort ratio during warmup ensures the system has already stabilized under the target conditions when steady-state measurement beginsDescribe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgement based on the scope of the change.

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users?
• If we are changing behavior how will we phase out the older behavior?
• If we need special migration tools, describe them here.
• When will we remove the existing behavior?

Test Plan

Describe in few sentences how the KIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

Rejected Alternatives

This feature is fully backward compatible:

• When --transaction-abort-ratio is not specified, the tool behaves exactly as before;
• No existing configuration options are changed or removed;
• All existing functionality remains intact.

Test Plan

Unit tests will cover the following scenarios:

1. Argument parsing: Verify that --transaction-abort-ratio 0.5 is parsed correctly
2. Range validation: Verify that values outside [0.0, 1.0] (e.g., -0.1, 1.5) throw an error
3. Transaction dependency validation: Verify that setting a non-zero abort ratio without enabling transactions throws an error
4. Abort logic:
   • With ratio 0.0, all transactions are committed (commitTransaction() is called)
   • With ratio 1.0, all transactions are aborted (abortTransaction() is called)
   • With intermediate ratios, verify that the number of commit and abort calls matches the expected proportion

Rejected Alternatives

NoneIf there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.