Status: Draft

Author: Viquar Khan (Vaquar.khan@gmail.com)

Discussion thread: TBD

Motivation The open-source community is currently seeing a massive increase in automated pull requests. While code generation tools can be helpful to developers, they frequently submit pull requests that look correct on the surface but are actually full of empty boilerplate, unnecessary comments, and hallucinated logic. This puts a heavy burden on our maintainers, who have to spend their limited time reviewing code that has no real substance.

Other major Apache projects are already dealing with this issue and taking action. For example, Apache Airflow recently had to change their contribution policies because automated bots were taking over open issues, and Apache Iceberg recently had to write strict new contribution guidelines to protect their reviewers.

Why AGENTS.md is not enough: While adding an AGENTS.md or CLAUDE.md file to the repository is a great first step to instruct well-behaving bots, it acts only as a "soft control". AI models use probabilistic reasoning; if their context window gets too full, or if a user overrides the prompt, the agent will silently ignore the markdown instructions and submit the bad code anyway. We cannot rely on the "good intentions" of an AI model to protect our codebase. We need a deterministic "hard control" to catch these failures before they consume a human reviewer's time.

Rather than waiting for our reviewers to burn out, we should be proactive. I propose we add custom, native validation logic to our Gradle build system to mathematically check the quality and substance of a pull request before a human reviewer is ever notified.

Public Interfaces This KIP introduces no changes to the Kafka protocol, public APIs, client behaviors, or broker metrics. It strictly modifies the project's internal build tools and continuous integration (CI) workflow.

Proposed Changes I propose we add two new custom Java-based validation tasks directly into Kafka's buildSrc / build-logic directory. By building this directly into our Gradle configuration, the checks run entirely locally without needing external API keys, introducing zero supply-chain risk, and allowing developers to run them on their laptops just like our existing ./gradlew checkstyleMain tasks.

These tasks will provide a two-layered defense against low-quality automated contributions:

1. Logic Density Validation (LDR) Task AI tools are notorious for generating massive pull requests with very little actual code, such as generating 12 lines of sophisticated documentation for a single line of logic. I propose adding a Gradle task that calculates a "Logic Density Ratio" (LDR), which mathematically measures the amount of logical or executable statements relative to the overall size of a code block. It evaluates the ratio of actual working code against the amount of boilerplate and comments. If a pull request submits 300 lines of setup and documentation but only 2 lines of actual logic, the task will flag it as low-effort scaffolding and fail the build.

2. AST Architecture Linter Task Automated tools often fail to understand Kafka's specific design paradigms. I propose adding a deterministic Abstract Syntax Tree (AST) parser task to systematically enforce our architecture based on our existing contributor guidelines.

Examples of problems solved:

• Concurrency Rules: KafkaConsumer is explicitly not thread-safe. Automated tools often make the mistake of wrapping it in standard java.util.concurrent threads. This AST check will automatically parse the syntax tree to detect and block thread-pool wrapping around consumer instances.

• Configuration Rules: AI agents frequently assume Kafka is a standard database and introduce dangerous defaults. The parser will block pull requests that blindly set enable.auto.commit=true (which leads to data loss during failures) or leave critical metrics like retention.ms at unsafe defaults.

• Code Standards: It enforces the usage of the final keyword wherever possible and ensures new public APIs have JavaDocs, as mandated by the current Kafka contributor guide.

Compatibility, Deprecation, and Migration Plan This proposal does not change the Kafka protocol or public APIs. It only updates our internal Gradle build logic and GitHub Actions workflow.

To ensure this does not slow down regular developers, the proposed tasks will include built-in exceptions:

• Refactor Exception: If a developer is cleaning up code and deleting more lines than they add (net negative lines), the Logic Density check is automatically skipped so legitimate refactoring is never blocked.

• Emergency Override: Anyone can add a /skip-validation flag to their commit message to bypass the checks during an urgent hotfix.

Test Plan To ensure this solution is robust and does not disrupt developer velocity, we will implement the following testing strategy:

1. Unit Testing the Gradle Tasks: The new logic in buildSrc will include its own JUnit tests. We will add test cases containing known "AI slop" patterns and concurrency violations to mathematically prove the AST parser and LDR calculator accurately reject bad code while allowing valid code.

2. CI Shadow Mode: We will implement these Gradle tasks and run them in a non-blocking "Shadow Mode" in our CI pipeline for 30 days. It will log its findings on incoming pull requests without actually failing the build or notifying contributors. The community can review these logs to ensure there are no false positives before we decide to make it a mandatory, blocking check.

Documentation We will update the CONTRIBUTING.md file to explain the new Gradle checks and instruct contributors to run ./gradlew checkPullRequestQuality locally before submitting a PR.

Rejected Alternatives

• Relying only on human reviewers: Catching low-effort automated submissions and concurrency mistakes manually does not scale and will lead to maintainer fatigue.

• Using third-party GitHub Actions: Using pre-compiled external binaries introduces supply-chain security risks and prevents developers from running the exact same checks locally on their laptops. Building it natively into our Gradle buildSrc solves both issues.