Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Flink has been using Jekyll for its user documentation since its earliest days. Over the past several years, as the community has grown, we have run up against several limitations with the current setup. In no particular order, these issues include:

• Slow Builds
  • A clean build can take > 5 minutes and an incremental build > 10 seconds
• Difficult build process
  • Requires docker or specific installed ruby dependencies
• Poor support for internationalization
• Broken search

These issues make it difficult for community members to contribute documentation, the first resource a new user sees when investigating Flink.

Proposed Changes

Migrate the docs in `flink/docs` with hugo a modern build tool. You can view a proof of concept of this migration here. Please follow the steps in the README to build the site locally.

We chose Hugo for a number of reasons.

• Hugo is fast
  • In local benchmarks the Flink docs build in less than 1 second

• Hugo is actively maintained by a large community
• Hugo supports all required functionality
  • Embed Flink's autogenerated configurations
  • Code blocks
  • Tabs for multi-language examples
  • Data Pages for generating SQL downloads
• There are many community resources available if we would like to add advanced functionality to the Flink docs
• Hugo has been adopted by other large open source projects including Kubernetes and Bootstrap
• Hugo is a single native binary
  • No complex installs or docker containers required
• Strong support for internationalization
• Working Search which is properly internationalized.
• Files are just normal markdown
  • Most documentation contributions will not need to look at HTML or JavaScript

IMPORTANT: this FLIP does not propose any changes to the content of Flink's documentation or Flink Web. Just the build tool for `apache/flink/docs`

We also chose to use this time to refresh the Flink docs UI. The goal is not a complete redesign but to modernize this look.

Homepage

Search

Misc page with code

Translate page

Migration

We have built a migration tool that is capable of rewriting most pages of the Flink docs. We will migrate all content and set up proper aliasing for any pages that move to avoid breaking changes. During the migration we will ask the community for a short documentation freeze to not miss any changes. We will perform the migration over the Chinese new year when many committers are on vacation to make this change as unobtrusive as possible.