The Flink documentation is build nightly via Github Actions.
The Flink(-StateFun) /-ML documentation is build nightly in the Apache buildbot (https://ci2.apache.org/) environmentenvironment.
The content is pushed to nightlies.apache.org/flink.
Build Environment
Github Actions
The following files configure the workflow for building the docs. Check it out for adjusting the build configuration.
Apache Buildbot
The following files configure buildbot for checking out branches and building the docs. Check it out for adjusting the build configuration.
- flink-statefun.conf: https://github.com/apache/infrastructure-bb2/blameblob/master/flink-statefun.py
- flink-statefunml.conf: https://github.com/apache/infrastructure-bb2/blob/master/flink-statefunml.py
Branch configuration
The configuration files above contain a flink*_branches_nightly variable that configures the branches to build nightly:
...
# format: (branch, alias) or branch
flink_statefun_branches_nightly = [
('master', 'release-2.1'),
('release-2.0', 'stable')
]
or, for Flink ML:
# format: (branch, alias) or branch
flink_ml_branches_nightly = [
('master', 'release-2.1'),
('release-2.0', 'stable')
]
The Flink docs will be served under the following URLs:
...
- https://nightlies.apache.org/flink/flink-statefun-docs-BRANCH (e.g. flink-statefun-docs-master)
- https://nightlies.apache.org/flink/flink-statefun-docs-ALIAS (e.g. flink-statefun-docs-stable)
The Flink ML docs will be served under the following URLs:
- https://nightlies.apache.org/flink/flink-ml-docs-BRANCH (e.g. flink-ml-docs-master)
- https://nightlies.apache.org/flink/flink-ml-docs-ALIAS (e.g. flink-ml-docs-stable)
Triggering a build manually
Navigate to github actions and click "run workflow", then select a branch.
Triggering a build manually CURRENTLY BROKEN
You can manually trigger a build via an IRC buildbot.
...
The command in the 3rd step will force build the docs of the provided builder, e.g. flink-docs-release-1.5 or flink-statefun-docs-release-2.0 in the above example.
[09:56] == uce [02cd5693@gateway/web/freenode/ip.2.205.86.147] has joined #flink-testing
[09:57] <uce> flink-buildbot: force build flink-docs-release-1.5
[09:57] <@flink-buildbot> build #7 forced
[09:57] <@flink-buildbot> I'll give a shout when the build finishes
Builder status
For Flink, you can check the progress of the build at, for example, https://ci2.apache.org/#/builders/17/builds/1.
For StateFun, you can check the progress of the build at, for example, https://ci2.apache.org/#/builders/14/builds/43.
For Flink ML, you can check the progress of the build at, for example, https://ci2.apache.org/#/builders/103/builds/2.
You can check the progress of all builders at https://ci2.apache.org/#/builders.
...
Note that directories containing an index.html
file cannot be deleted; you first need to delete said file before deleting the directory.
Currently it is not possible to delete directories/files uploaded via Github Actions.
Instead one needs to run a workflow that deletes the directory via rsync.
As this is incredibly error prone with fatal consequences (i.e., wiping the entire documentation for all versions), this should NOT be attempted unless you absolutely know what you're doing.
Manual upload
As an emergency measure it is possible to manually upload files to nightlies.apache.org
, as explained in https://nightlies.apache.org/authoring.html.
...
net-rc-file contents: machine nightlies.apache.org login <username> password <password>
Testing changes
Github Actions:
Modify the workflow to skip the rsync steps and trigger a manual build.
Buildbot:
Changes to the website build process can be tested by pushing to the docs_experimental__docs
and docs_experimental__docs_compile
branches (of the Flink repository) and triggering a manual build. The first branch will only run build_docs.sh
, while the latter will also compile Flink and generate the java/scala/python docs.
...
This section contains instructions for building the documentation of older Flink versions.
Flink 1.0 -> 1.7:
docs/build_docs.sh
:
Code Block |
---|
-bundle install --path .rubydeps +bundle install --full-index --path .rubydeps |
...
Code Block |
---|
docker run --rm --volume="$PWD:/srv" --expose=4000 -p 4000:4000 -it ruby:2.3.1 bash -c "cd /srv && ./build_docs.sh" |
Flink 1.8 -> 1.12
...
/ StateFun 2.0 -> 2.2:
Code Block |
---|
docker run --rm --volume="$PWD:/srv" --expose=4000 -p 4000:4000 -it ruby:2.6.3 bash -c "cd /srv && ./build_docs.sh" |
...