Status

Current state: Under Discussion

...

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

For Kafka, an SSL connection occupies approximately 100KB of memory, while a plaintext connection occupies around 250 bytes, resulting in a memory footprint ratio of approximately 400:1. Therefore, there should be a limitation for SSL connections in broker wide to prevent potential OOM situations caused by an excessive number of SSL connections.

...

The new configuration could work together with max.connections to control ssl and non-ssl connections. For examplem, set max.connections to 10000, max.ssl.connections to 5000, which means non-ssl connection radio is set at 5000.

Public Interfaces

No new interfaces or will be added.

...

Config option: Name: max.ssl.connections Type: Int Default value: Int.MaxValue

Proposed Changes

Acceptor will stop accepting new connections when the broker's max.connections limit is reached. New connections will be accepted as soon as a connection is closed by any of the Processors. Acceptor will also stop accepting new connections when its listener's listener.name.{listener}.max.connections limit is reached. New connections will be accepted as soon as a connection is closed by any of the Processors of that listener. Inter-broker connections will be protected in multi-listener brokers by closing client connections to accommodate inter-broker connections. Any time spent by Acceptor waiting for connections to close will also be included in the new AcceptorBlockedPercent metric. The existing max.connections.per.ip config will be applied without any changes. Connections dropped due to hitting the per-ip limit will not appear in the AcceptorBlockedPercent metric since these connections are accepted and then dropped.

...

I propose adding an additional condition: if the number of SSL connections exceeds the SSL connection limit and the current listener is using SSL protocol, the connections may be closed.

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users?

No externally visible interface changes are proposed in this KIP. During normal operations, this is unlikely to result in any impact. When a large number of ssl connections are made to the broker at the same time, connections may be established slower, and the least recently used connections may be disconnected.

Test Plan

Will be tested in org.apache.kafka.network.ConnectionQuotasTest.scala in the following cases:

• This configuration could be updated dynamically and correctly
• Hiting the limit of max.ssl.connections, new ssl connections will be blocked, while plaintext listener will not be affected.
• Removing one sasl connection should make the waiting connection to succeed
• sslConnectionCounts variables updated correctly

Rejected Alternatives

It might be argued that the existing listener-specific limit already provides satisfactory ability to limit SSL listener.

As mentioned above：

1. Unable to limit ssl connections precisely.
2. The protocol of the listener may change dynamically, and the limit of the listener also needs to be modified.

...