...

This page is meant as a template for writing a KIP. To create a KIP choose Tools->Copy on this page and modify with your content and replace the heading with the next KIP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: [One of "Under Discussion", "Accepted", "Rejected"]

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread] 

JIRA: here [Change the link from KAFKA-1 to your own ticket]

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Describe the problems you are trying to solve.

Public Interfaces

Briefly list any new interfaces that will be introduced as part of this proposal or any existing interfaces that will be removed or changed. The purpose of this section is to concisely call out the public contract that will come along with this feature.

A public interface is any change to the following:

• Binary log format

• The network protocol and api behavior

• Any class in the public packages under clientsConfiguration, especially client configuration

  • org/apache/kafka/common/serialization

  • org/apache/kafka/common

  • org/apache/kafka/common/errors

  • org/apache/kafka/clients/producer

  • org/apache/kafka/clients/consumer (eventually, once stable)

• Monitoring

• Command line tools and arguments

• Anything else that will likely break existing users in some way when they upgrade

Proposed Changes

...

Motivation

The current Apache Kafka documentation website (https://kafka.apache.org/documentation/) uses raw HTML embedded within the source code. This approach presents several challenges:

• Inefficient Maintenance: Editing raw HTML is cumbersome and mixes content with styling, making updates difficult and increasing the barrier to contribution, especially for developers unfamiliar with HTML/CSS. Testing changes requires deploying to a web server.
• Styling Inconsistencies: The site exhibits inconsistencies in styling, such as inconsistent heading levels across different pages (e.g., some pages start with H2 headings, while others use H4).
• Long Pages: Several pages are excessively long, impacting readability and maintainability.
• Server-Side Dependencies: The website relies on server-side includes (SSI) for dynamic page generation, which introduces potential security vulnerabilities. For example, improper configuration of SSI can lead to information disclosure or cross-site scripting (XSS) attacks. More details on SSI vulnerabilities can be found in resources like OWASP's documentation on Server-Side Includes injection. The site also uses Handlebars.js as a templating engine, further complicating the structure as raw HTML is embedded within <script> tags.

This KIP proposes migrating the Apache Kafka documentation website from its current raw HTML format to Markdown. This change will leverage modern static site generation tools like Hugo and the Docsy theme to improve maintainability, readability, and testability of the documentation, while also enabling richer features and a more consistent user experience.

Public Interfaces

N/A since there are no changes to Apache Kafka codebase.

Proposed Changes

I propose migrating the Apache Kafka documentation website to Markdown and using Hugo (https://gohugo.io/) for static site generation. I also propose leveraging the Docsy theme (https://www.docsy.dev/docs/get-started/), which is used by other successful open-source projects like Kubernetes (https://github.com/kubernetes/website/tree/main/content) and Istio (https://github.com/istio/istio.io/tree/master/content). This approach offers several advantages:

• Modern Tooling: Markdown offers a simpler, more readable, and maintainable format compared to raw HTML. Hugo provides a powerful and efficient static site generator with a rich ecosystem of tools and themes.
• Improved Maintainability and Testability: Markdown's simplicity makes it easier to write, edit, and review documentation. Static site generation simplifies testing as the entire site can be built locally without requiring a live web server.
• Richer Features: Using Hugo and Docsy opens up access to a wide range of features, including improved navigation, local search functionality, better mobile responsiveness, and potential integration with other Kafka resources.
• Content Refactoring: This migration provides an opportunity to refactor the content for better organization and readability. As detailed in the automation repository (https://github.com/hvishwanath/ak2md) and the process.yaml file (https://github.com/hvishwanath/ak2md/blob/main/process.yaml), specific refactoring tasks are planned. For example, the current documentation includes several long pages. These will be split into smaller, more manageable sections. Additionally, the information about Kafka Connectors will be reorganized to improve the user experience. The current documentation also has some inconsistencies in the structure of the quickstart guides. This will be standardized as part of the migration to Markdown.

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users?
• If we are changing behavior how will we phase out the older behavior?
• If we need special migration tools, describe them here.
• When will we remove the existing behavior?

Test Plan

Describe in few sentences how the KIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

Rejected Alternatives

...

• Compatibility: This change is not expected to impact the compatibility of Apache Kafka itself. It only affects the documentation website.
• Deprecation: The existing HTML-based documentation website will be deprecated once the new Markdown-based version is launched.
• Migration Plan:
  1. Complete the conversion of any remaining HTML documentation to Markdown.
  2. Thoroughly test the migrated documentation to ensure accuracy and consistency.
  3. Deploy the new documentation website.
  4. Update any relevant links or references to the documentation.

Test Plan

• Unit Tests: The individual Markdown files will be checked for valid Markdown syntax.
• Integration Tests: The Hugo build process will be tested to ensure the site is generated correctly.
• End-to-End Tests: The deployed website will be tested for functionality, usability, and accessibility.

Rejected Alternatives

• Maintaining the Status Quo: Continuing with the current raw HTML approach is not sustainable due to the issues outlined in the "Motivation" section.
• Alternative Static Site Generators: While other static site generators exist, Hugo was chosen for its popularity, maturity, and strong community support. The Docsy theme aligns well with the needs of technical documentation.