You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Status: Draft

Author: Viquar Khan (Vaquar.khan@gmail.com)

Discussion thread: [Link to your dev mailing list thread]

Motivation The open-source community is currently seeing a massive increase in automated pull requests. While code generation tools can be helpful to developers, they frequently submit pull requests that look correct on the surface but are actually full of empty boilerplate, unnecessary comments, and made-up dependencies. This puts a heavy burden on our maintainers, who have to spend their limited time reviewing code that has no real substance.

Other major Apache projects are already dealing with this issue and taking action. For example, Apache Airflow recently had to change their contribution policies because automated bots were taking over open issues without delivering actual work. Apache Iceberg also had to write strict new contribution guidelines to protect their reviewers.

Rather than waiting for our reviewers to burn out, we should be proactive. I propose we add a set of native, automated quality scripts to our continuous integration (CI) pipeline. This will mathematically check the quality and substance of a pull request before a human reviewer is ever notified.

Public Interfaces This KIP introduces no changes to the Kafka protocol, public APIs, client behaviors, or broker metrics. It strictly modifies the project's continuous integration (CI) workflow on GitHub.

Proposed Changes I have written a set of lightweight, open-source Python scripts that I would like to donate and integrate directly into our GitHub Actions. These scripts run entirely locally, require no external API keys, and provide a defense against low-quality automated contributions:

1. Logic Density Validation (LDR) AI tools are notorious for generating massive pull requests with very little actual code. I have written a script that calculates a "Logic Density Ratio" (LDR), which measures the amount of logical or executable statements relative to the size of a code block. It evaluates the ratio of actual working code against the amount of boilerplate and comments. If a pull request submits 300 lines of setup and documentation but only 2 lines of actual logic, the CI pipeline will flag it as low-effort scaffolding and block it before it wastes a maintainer's time.


2. AST Architecture Linter Automated tools often fail to understand Kafka's specific concurrency rules. I have built a deterministic Abstract Syntax Tree (AST) parser that uses a simple design-rules.yaml file to enforce our architecture. For example:

  • Concurrency Rules: KafkaConsumer is explicitly not thread-safe, and automated tools often make the mistake of wrapping it in standard java.util.concurrent threads. This script will automatically block that exact mistake.

  • Configuration Rules: It will block pull requests that try to blindly set enable.auto.commit=true (which leads to data loss) or leave critical metrics like retention.ms at unsafe defaults.

  • Code Standards: It enforces the usage of the final keyword wherever possible and ensures new public APIs have JavaDocs, as mandated by our contributor guide.

3. Contributor Instructions (AGENTS.md) We will add an AGENTS.md file to the root of our project. This acts as a clear instruction manual for automated tools, telling them our formatting rules and our requirement to disclose automated code with a "Generated-by" token in the commit message before they even write the code.

Proposed Architecture Workflow

Code snippet

graph TD
    A --> B{Is Core Maintainer?}
    B -- Yes --> C
    B -- No --> D{Commit contains '/skip-validation' flag?}
    D -- Yes --> C
    D -- No --> E{Dependency Gate}
    E -- Fails --> F
    E -- Passes --> G{Logic Density Gate}
    G -- Fails --> H
    G -- Passes --> I{AST Design Gate}
    I -- Fails --> J
    I -- Passes --> C

Compatibility, Deprecation, and Migration Plan This proposal does not change the Kafka protocol, public APIs, client behaviors, or broker metrics. It only updates our GitHub Actions workflow.

To ensure this does not slow down regular developers, the scripts include built-in exceptions:

  • Refactor Exception: If a developer is cleaning up code and deleting more lines than they add (net negative lines), the Logic Density check is automatically skipped so legitimate refactoring is never blocked.

  • Emergency Override: Anyone can add a /skip-validation flag to their commit message to bypass the checks during an urgent hotfix.

Test Plan We will run these scripts in a non-blocking "Shadow Mode" for 30 days. It will log its findings on incoming pull requests without actually blocking them or notifying contributors. The community can review these logs to ensure there are no false positives before we decide to make it a mandatory check.

Documentation We will update the CONTRIBUTING.md file to explain the new continuous integration checks and document how the AGENTS.md file works.

Rejected Alternatives

  • Doing nothing: Relying entirely on human reviewers to catch low-effort automated submissions and concurrency mistakes does not scale and will lead to maintainer fatigue.

  • Restricting pull requests to collaborators only: While this stops bot spam natively in GitHub, it heavily restricts legitimate first-time contributors and contradicts the open-source ethos.

Subject: SPIP: Automated Pull Request Validation and Architectural Compliance

Q1. What are you trying to do? The Apache Spark project is seeing an increase in automated pull requests. These submissions often pass standard syntax checks but contain padded boilerplate, made-up dependencies, and excessive comments without real logic. I am proposing we add a set of custom, native continuous integration (CI) scripts to mathematically filter out this low-effort code and catch common Spark coding mistakes before a human maintainer reviews the code.

Q2. What problem is this proposal NOT designed to solve? This is not designed to replace human review for complex architecture decisions. It is also not designed to ban automated tools for legitimate human contributors. It only filters out zero-substance submissions and catches known API mistakes natively in GitHub Actions.

Q3. How is it done today, and what are the limits of current practice? Right now, Spark maintainers rely on manual review and standard compiler checks. A pull request with 400 lines of useless documentation and a highly inefficient withColumn loop will pass basic CI tests. A human committer then has to waste time explaining why the code is structurally flawed.

Q4. What is new in your approach and why do you think it will be successful? I have written a set of lightweight validation scripts that I want to donate to the project's CI pipeline. The two main features are:

  • Logic Density Ratio (LDR): A mathematical formula that checks if a pull request has enough actual execution logic compared to its scaffolding.


  • AST Architecture Linter: A parser that uses a design-rules.yaml file to scan the code structure and enforce Spark's evolving API standards.

Q5. Who cares? Project maintainers and committers who are suffering from review fatigue due to an influx of low-quality, automated pull requests.

Q6. What are the risks? The main risk is a "false positive" where a legitimate pull request is blocked. We prevent this by including a "Refactor Exception" that automatically skips the density check if a developer is deleting code (net negative lines). We also allow trusted maintainers to bypass the checks with a simple commit flag.

Q7. How long will it take? Adding the scripts to our GitHub Actions and writing the initial design-rules.yaml will take about two weeks of testing and feedback on the developer mailing list.

Q8. What are the mid-term and final “exams” to check for success?

  • Mid-term: Run the new checks in a non-blocking "shadow mode" for one month to monitor accuracy.

  • Final: Achieve a measurable drop in the time maintainers spend reviewing low-quality pull requests.

Appendix A. Proposed Design Rules Examples The AST linter will actively scan the code for common mistakes that automated tools frequently make:

  1. Production Mistakes: Flagging pull requests that try to use display(), collect(), or print(df.count()) in core execution paths.

  2. Loop Mistakes: Automatically failing pull requests that place withColumn() inside a for loop, prompting the contributor to use select() with multiple columns instead.

  3. Deprecation Checks: Rejecting code that introduces outdated configurations.

  4. Partitioning Mistakes: Catching instances where data is written to disk formats without using proper partitioning strategies like .partitionBy().

  • No labels