Status

StateCompleted
Discussion Thread[PROPOSAL] Add a landing page for Apache Airflow
JIRA

Unable to render Jira issues macro, execution error.

Created

$action.dateFormatter.formatGivenString("yyyy-MM-dd", $content.getCreationDate())

In Release1.10.7

Motivation

Open Source Software needs to build a community to ensure its healthy growth. One of the tools used to build a community is the web site dedicated to the product. The goal behind building an open source friendly website for Apache Airflow is to engage, inform, and educate visitors, and ultimately to convert visitors into users and users into contributors, thus increasing the adoption of the product.

Introduction

This document overviews the general requirements for Apache Airflow website. It includes a list of necessary functions, capabilities, and/or characteristics related to Apache Airflow website, as well as the plans for creating it.

Apache Airflow website, like the rest of the project, will be open source, and accept community contributions. It will make contributions easy and approachable, while enforcing a set of best practices for developing the website and its content.

Improvements will also include a contributor’s guide that easily introduces prospective contributors to the community, and the process for starting to contribute to the project. It will also clarify the different kinds of contribution areas, such as  code, community development, project vision, documentation, bug reports, feature requests and any other constructive contributions.

Knowledge Architecture

OSS projects’ websites typically include several standard pages, where each page is formatted with a navigation bar on the left.

Taking from best practices and based on other OSS websites -- Apache Beam, Apache LibCloud, and others. A preliminary knowledge architecture for the website is provided in the next section. We outline important pages that are must-haves for OSS websites. As we advance in our project development, we will iterate the list and include necessary components that may be missing.

Home Page

Apache Airflow’s Home Page is the primary entry point to the site that contains project description, news, invitation to join the project. The homepage is divided into three sections that users can navigate by scrolling it down. Each section goes into further detail about Apache Airflow’s features, usability, and contributions. The structure is presented below. (For a more detailed description of the website structure see the appendix).

Section 1

  • Title

  • Short description

  • Link to the Quick start or (Learn Apache Airflow basics) which will be cross-linked with Documentation

  • Downloads

  • Announcements (can be cross-linked with Community)

Section 2

  • Apache Airflow features

Section 3

  • Use cases

  • Testimonials?

  • Communication channels, site map

Other must-have pages:

  • License Page: mention the Apache License 2.0 and link to the git license file

  • Downloads: page where Apache Airflow will release code. Downloads page describes the release/update/features and has links to the download pages that redirect to https://Apache Airflow/downloads/

  • Documentation: this page describes the project documentation, including guides, tutorials, and links to external documentation.

  • Mailing Lists: page will include user@, dev@, and commits@ mailing lists that the community might be interested in, and this page contains mailto: links that allow easy subscription (and unsubscription) to any of them. Private@ mailing list can be listed as well, but the subscription won’t be available.

  • FAQ: this page will include frequently asked questions and answers to them.

  • Road Map: this page will have a description of the project’s vision of future community or development activities.

  • Source Code: page will have links to the browsable source repository and git commands to check out the sources.

  • Coding Standards: page will include a description of the coding standards for submitted code by the community, along with a description of how strict the project intends to be.

  • Issue Tracking: page will have links to the JIRA or other issue tracking tool.

Proposed organization of must-have pages in Top bar items:

  • About

    • A Brief History of Apache Airflow, license

  • Documentation (kept in Pydocs)

    • Guides, tutorials, and links to external documentation

    • FAQ

  • Community

    • Roadmap

    • Events and meetups

    • Mailing lists, slack channels, interest groups

    • Jira bug tracker, StackOverflow

    • Code of Conduct

  • Contribute

    • Contributor guides

    • Contributing Code

  • Blog

  • Downloads

Website Generation Tools

Several options are being considered as a tool for generating Apache Airflow website, such as Hugo, Jekyll, etc.

Regardless of which tool we use, the website should be maintained in the git repository, and include the site generation tool as a binary file. This simplifies the process of site generation and enables changes to the site to be made by any committer.

Since the site is independent of the code, it should exist high in the git repository, e.g. parallel to the trunk of the source tree.

Content generation

Content of the website will include both auto-generated Pydocs for Documentation page and static front-end pages for the rest of the website.

All content will be developed and added by contributors.

Server and Hosting

Needs to be open source friendly - source code on GitHub along with the codebase

  • GitHub ?

  • “Netlify” claims to be free for OSS.

Testing

A number of tests, and checks should be written to ensure that changes and contributions to the website are of high quality, and that its appearance will not break or be disrupted by new contributions.

  • Automatic staging of changes for review

  • Checking for dead links

  • Possibly a spell checker

Action Items

The following is a comprehensive list of high-level action items.

  • Create a directory in the Apache Airflow repository to house the website

  • Determine the tool to be used to generate the website (Hugo, Jekyll, ..)

  • Create the knowledge architecture of the website

  • Create and receive approval for the design of the website

  • Create and receive approval for an initial prototype for the website

    • This prototype can be committed into the GitHub repository

  • Iterate to build an MVP

  • Move pages from Sphinx documentation to the new website

    • Team members, contributor’s guide, etc.

  • Build a set of tests for the website

  • Build a script to host and update the website

  • Change the website to the new landing page, and start receiving PRs from the community


Scope of Work

The project is starting on 8/26 and planned to be finished on 11/1.

Milestone 1: UX Design

  • Interactive prototype in InVision with the wireframes of the website (desktop version)
  • .png files of the wireframes for desktop, mobile, and tablet
    • The landing page including the site navigation menu
    • About, Documentation, Downloads, Community, Contribute, Resources (Case Studies, Videos, Tutorials)  pages
  • Brand book with updated logo (including logo safe space, colors, how not to use the logo and black&white version) - up to 7 pages, PDF

Acceptance criteria: 

  • The design of the website is accepted by Apache Airflow community and corresponds to all requirements of an open-source project design requirements. Should we not receive an acceptance or feedback from Apache Airflow community within 72 hours we will rely on Lazy Consensus rule. 

Milestone 2: User Testing & Evaluation

  • Key findings from the user testing in a presentation (PDF)
    • Including 3 male and 3 female users
  • Correcting the wireframes based on feedback from the community and user feedback
    • .png of the wireframes for desktop, mobile, and tablet

Acceptance criteria:

  • N/A

Milestone 3: UI Design

  • Exported mockups to Zeplin for desktop, mobile, and tablet
    • The landing page including the site navigation menu
    • About, Documentation, Downloads, Community, Contribute, Resources (Case Studies, Videos, Tutorials)  pages
    • Icons for feature descriptions
    • Updated prototype in inVision to share with the community

Acceptance criteria:

  • The prototype of the website is accepted by Apache Airflow community and corresponds to all requirements of an open-source project design requirements. Should we not receive an acceptance or feedback from Apache Airflow community within 72 hours we will rely on Lazy Consensus rule. 

Milestone 4: Website Development

  • The landing page including the site navigation menu according to the design 
  • Cut layout into HTML/CSS
  • Responsive web design
  • About, Documentation, Downloads, Community, Contribute, Resources (Case Studies, Videos, Tutorials)  pages according to the design
  • Developed static website with tools accepted by community
  • Bug fixes and improvements after review from Apache Airflow community
  • Documentation for Apache Airflow community to maintain the website in future

Acceptance criteria: 

  • Website developed using tools accepted by Airflow community 
  • Automatic content tests: spell checker and dead links checks
  • Automatic code tests e.g. check syntax/code style.
  • Documentation for making look and feel changes

Milestone 5: Documentation redesign, Sphinx integration, setup CI/CD

  • Implementation of custom Sphinx theme according to new design
  • Building script and integration with main site
  • Automatic deploy of changes for review
  • Establish and document a process for changes using pull requests
  • Precise documentation on maintenance of both content and operation of the website

Acceptance criteria: 

  • The documentation page is accepted by Apache Airflow community and corresponds to all requirements of an open-source project design requirements. Should we not receive an acceptance or feedback from Apache Airflow community within 72 hours we will rely on Lazy Consensus rule. 

Milestone 6: Finalizing feedback, bug-fixing and testing

  • Necessary UI adjustments to the final website version
  • Knowledge transfer and additional documentation such as README.md and dev/user docs

Acceptance criteria:

  • The complete website (static website and documentation) is accepted by the community

References

Appendix. Detailed structure

Homepage

Section 1

  • Title

    • Apache Airflow

  • Short description

    • One or two short sentence(s) to describe the product, e.g. “Airflow is a platform to programmatically author, schedule and monitor workflows.”


The same page MUST have links to:

  • Quick start or (Learn Apache Airflow basics) which will be cross-linked with Documentation and include:

    • Overview

    • Tutorials

    • Examples

    • Guides

    • Videos

  • Downloads

    • Links to download different version of the product

    • Features and improvements

  • Announcements (can be cross-linked with Community)

    • Upcoming events

    • Meetups

Section 2

  • Apache Airflow features

    • Include icon and description

Section 3

  • Use cases

    • Companies that use Apache Airflow

    • Testimonials

Top bar items

  • About

  • Documentation, which will link to the existing https://airflow.apache.org/ website.

  • Community

    • Airflow contributor Code of Conduct

    • Stack Overflow

    • Twitter

    • GitHub

    • Slack

    • Roadmap

    • Code of conduct

    • Meetup schedules/events (cross-linked with homepage)

    • Mailing lists

    • Other communication channels

  • Blog

    • The blog will be edited via source control

    • Content from the Apache Airflow team

    • Articles from the community


  • Contribute (can be cross-linked to Documentation or Community).

Cover:

    • Comprehensive contributor guides

    • Ways to contribute

            Include:

    • Start contributing

      • Repositories

      • Prerequisites/Contributor checklist

  • Downloads / Releases (cross-linked with Homepage download)

    • Contributor recognition?

    • New version of the product

    • Features and improvements

  • Site search - maybe

    • Try to measure feasibility of Apache Airflow site searching capabilities

    • User is able to find content

Footer

  • Communication channels

  • Sitemap