Table of Contents |
---|
Proposers
...
Current state: Status colour BlueYellow title INactiveIn progress
Discussion thread: here
JIRA: Jira server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key HUDI-504
...
To address these gaps, restructuring of the docs is needed to make the process easier. Migration with Travis-ci, it can build the asf-site branch and execute callback scripts.
The diagram below shows the structuring and the workflow.
drawio
File structure
Eventually, we will have all .md of docs files on master, thus allowing the docs change with code change to be committed in a single PR. The automation scripts to generate the docs content will also be on master. To start with, we still keep all docs files on a separate branch, i.e., asf-site.
Versioning docs
The documentation should evolve with the code changes. Thus, we need to version the docs regarding design, implementation details and API examples according to each release version. In this way, the code and the corresponding docs change can coexist in the same PR. Specific pages around these include:
- Quickstart
- Concepts
- Writing Data
- Querying Data
- Configuration
- Performance
- Administering
In addition to these content, a new set of API docs will be generated by javadoc for each release (similar to this).
Content regarding the general information of Hudi only have one version, generated from the latest changes:
- Use Cases
- Talks & Powered By
- Comparison
- Releases
- Community
- Code
- Developers
- Feedback
The docs content are still generated from a single branch, so that all page references can be generated properly. The generated web pages are then uploaded to the hosting server for the Hudi site. The landing page references the specific version of docs content through links.
For backward compatibility, we can manually generate the content for current (0.5.0-incubating) and one or two old releases (0.4.6/0.4.7), to fit into the new docs structure.
Automating docs update
To make updating and deploying docs content easier, a set of scripts automating the above process will be added (including sanity checks of whether there are breaking changes, e.g., broken links, missing pages, before actual changes to the site). In this way, once the changes to the /docs on either master or asf-site are landed, the update content pages can be uploaded to the hosting server of Hudi website.
Rollout/Adoption Plan
- No impact on existing users regarding APIs as this is docs website change.
Test Plan
- Test the content organization locally to make sure the links are working and the content is readable
Appendix
...
Key Points
There are several key points need to consider:
- How to migration with travis-ci
- How to use git command safely in travis-ci (important)
To use git command safely, hidden the password, we need use bellow form.
Code Block | ||
---|---|---|
| ||
git clone https://${GIT_TOKEN}@github.com/${GIT_REPO}/${GIT_PROJECT}.git |
Steps to get the {GIT_TOKEN}
Steps - 01 : Enable two-factor authentication
https://help.github.com/en/github/authenticating-to-github/about-two-factor-authentication
Steps - 02 : Generate personal access tokens
Steps to migration with travis-ci
Steps - 01 : Active repository
Steps - 02 : Add {GIT_TOKEN} to environment variables
Steps - 03 : Add .travis.yml file to asf-site branch
Code Block | ||
---|---|---|
| ||
language: ruby
rvm:
- 2.6.3
git:
clone: false
env:
global:
- GIT_USER="CI BOT"
- GIT_EMAIL="cibot@test.com"
- GIT_REPO="apache"
- GIT_PROJECT="incubator-hudi"
- GIT_BRANCH="asf-site"
- DOCS_ROOT="`pwd`/${GIT_PROJECT}/docs"
before_install:
- git config --global user.name ${GIT_USER}
- git config --global user.email ${GIT_EMAIL}
- git clone https://${GIT_TOKEN}@github.com/${GIT_REPO}/${GIT_PROJECT}.git
- cd ${GIT_PROJECT} && git checkout ${GIT_BRANCH}
- gem install bundler:2.0.2
script:
- pushd ${DOCS_ROOT}
- bundle install
- bundle update --bundler
- bundle exec jekyll build _config.yml --source . --destination _site
- popd
after_success:
- \cp -rf ${DOCS_ROOT}/_site/* test-content
- git add -A
- git commit -am "Travis CI build asf-site"
- git push origin asf-site --force
branches:
only:
- asf-site |
Steps - 04 : Check whether ci works well or not
End
Hope, this will works well
Test
had test https://github.com/lamber-ken/lamber-ken.github.io, when commits, it will generate docs and push the content to asf-site branch.