This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: Draft

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: here [Change the link from KAFKA-1 to your own ticket]

...

Kafka has a compile-time dependency on the AccessController  class in clients , core , and connect:runtime , and the removal of SecurityManager and it's accompanying classes would make these modules un-buildable with their current implementation.

Public Interfaces

There are multiple strategies for when to remove SecurityManager support. The following are listed from shortest to longest support period, but may overlap with one-another.

Below, "Java R" refers to the first Java version which has Removed the SecurityManager, and "Java R-1" is the last Java version which has the SecurityManager.

Option A: Remove SecurityManager support in the next major Kafka version (4.0), before support for Java 11 is dropped

Pros:

• Removes SecurityManager the earliest, allowing users with Kafka 4.x to upgrade to Java R as soon as it is released

Cons:

• Users on Java 11 (where SecurityManager was never deprecated) will experience a removal with this KIP as a prior deprecation notice

Option B: Remove SecurityManager support in the first major Kafka version which drops support for Java 11 (5.0+)

Pros:

• Users on Java 11 will not see a Kafka release without SecurityManager support

Cons:

• This may delay adoption of Java R if it is released before the Java 11 support is intended to be removed
• We will need to commit to removing support for Java 11 at some future version, like we did in KIP-750 for Java 8

Option C: Remove SecurityManager support in a major Kafka version that adds support for a Java R (5.0+)

Pros:

• This allows the project to keep a single source tree/artifacts for all Java versions

Cons:

• It will inconvenience users which do not use the SecurityManager when they try to upgrade to Java R without it being officially supported
• It may also force a major version bump when it is inconvenient for the project, and other efforts requiring a major version bump may not be ready for it
• This may force dropping support for Java 11 earlier than expected

Option D: Remove SecurityManager support in a major Kafka version which drops support for Java R-1 (6.0+)

Pros:

• This is the longest window for SecurityManager support, and would give users the most opportunity to find an alternative.

Cons:

...

The Kafka project should to define a major release in which SecurityManager support will be removed. It is not possible to remove support for this feature in a minor release, as it would constitute a breaking change.

There are two strategies for approaching the removal that we should pursue concurrently:

Remove SecurityManager support as soon as possible

As SecurityManager has not yet been removed from Java, we have the opportunity to remove it ourselves first.

For example, if SecurityManager is removed from the Kafka codebase in 4.0, and SecurityManager is removed from Java 25, users of Kafka 4.x can immediately upgrade to Java 25 without special considerations.

However, there are some problems with removing support quickly:

1. Java 11 still has the non-deprecated SecurityManager, and Kafka 4.0 must support Java 11. If we were to remove SecurityManager support from Kafka 4.0, users on Java 11 would experience a removal without a prior deprecation.
2. Java 17 does not have the replacement Subject API https://bugs.openjdk.org/browse/JDK-8267108 , and Kafka 4.0 must support Java 17. We would need dual support for the old and new APIs.

The earliest we could remove the SecurityManager without tech-debt is when Java 21 could become the minimum supported Java version. This is not planned for Kafka currently, but would probably be Kafka >=6.0.

Remove compile-time usages & detect SecurityManager removal at runtime

The longer it takes for Kafka to remove SecurityManager support, the more likely that the Java removal will take place before Kafka's removal is complete. To be prepared, we should anticipate that the project will need to support both versions with and without SecurityManager, using the same codebase & artifacts.

This can be done by replacing direct references to deprecated APIs with reflection, and gracefully changing the behavior for Java versions without the SecurityManager. When Java 21 becomes the minimum supported Java version, the reflective implementations could be removed and replaced with direct calls to the modern API.

...

Proposed Changes

The SaslClientCallbackHandler and OAuthBearerSaslClientCallbackHandler in clients  will no longer call Subject.getSubject , and instead will use the JDK 18+ Subject.current API: https://bugs.openjdk.org/browse/JDK-8267108 . TODO: this isn't good enough, because it means that JDK 17 has no way to provide a Subject to the handlers... Does this functionality need to be removed?
The SaslClientAuthenticator and SaslServerAuthenticator in clients will use the Java 18+ Subject.callAs API rather than the deprecated Subject.doAs function. This has a backwards-compatible implementation that will presumably be replaced when SecurityManager is removed.
The calls to getSubject, current, doAs, and callAs will be replaced with reflection. If Java 18 is available, the new functions will be used. Otherwise, the legacy API will be called.

The calls to AccessController.doPriviliged  in core  and connect:runtime:  used for ClassLoader instantiation will be removed, and replaced with reflection. If the doPriviliged call is not available, the privileged operations will be done in the original context.

...

Users upgrading to Java 17+ (in which SecurityManager was deprecated) already receive a warning message on startup. We can additionally include a warning log in the next minor release that SecurityManager will be removed in a future Kafka version, and inform users of the removal schedule specified in this KIP.

Test Plan

There are currently no tests exercising the SecurityManager interface in Kafka, and no new tests for the end-to-end behavior will be added for this project.

Rejected Alternatives

Further tests will compare the reflective implementations to the old and new APIs. This is to ensure that changing the legacy APIs to reflection, and that later when the reflection is replaced with the new APIs, the behavior will stay the same throughout. These tests can mock the AccessController and Subject classes to make it appear that the removal has taken place.

Rejected Alternatives

Wait until SecurityManager is removed to change the implementations (do nothing)

When the first Java version without SecurityManager is released, there will have already been a number of Kafka minor and patch releases made. All of these will be unable to run with the new Java version, regardless of if the user was using the SecurityManager at all. This also includes users of the clients, who will be unable to run in the latest Java version until the clients have been released again.

Similarly, developers will be unable to build older branches of Kafka with the new Java version, and this will be generally inconvenient. When we front-load the effort in this migration, we can ensure that when the removal takes place, the most recent Kafka releases are already compatible.

Since removing SecurityManager is a breaking change, we would have to either delay support for the new Java version significantly, or perform a major release of Kafka when we might not be prepared for it. These negatives outweigh the maintenance burden of making these function calls reflective.

Set a removal date for Java 11 / Java 17 support

This should be delegated to another KIP with a dedicated discussion. This KIP can be blocked on that one, or be conditional. For example, we can agree here that in the major release which removes Java 17 support, we will also remove SecurityManager support, but not specify when the removal of Java 17 support will take place.

Implement a alternative to the SecurityManager for use with Kafka

The SecurityManager is a flawed design, and we should not replicate it or try to solve those problems in a similar way. Instead should encourage users to use the process boundary as a security boundaryIf there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.