...

Note: everything here is just experimental which can help to decide which method can be the best for the future. This wiki page summarizes the current state/experiences to be base of a discussion.

Earlier we had Jenkins experiments see the history of this page if you are interested about the problems with Jenkins.

All the checks are committed to hadoop-ozone/dev-support/checks. You can run it locally or on any server. As of now I am running them on a kubernetes cluster with the help of argo workflow.

TLDR;

1.  You can create pull request to apache hadoop repository
2.  You need to add the ozone label to check it with Jenkinsthis CI (if you have no permission to do it add a comment with the text /label ozone).
3.  It will be checked by this Jenkins: https://ci.anzix.net/job/ozonea cluster of kubernetes nodes.
4.  The result will be visible as github status checks: See the examples here: published under https://github.com/apache/hadoop/pull/464elek/ozone-ci-q4
5.  The checks are additional checks, not replacement of the Yetus checks. AW and others are working on the PR support of Yetus.

The background

There are two main methods to check github PR-s in Jenkins:

1.   Github Branch source plugin + Pipeline: multibranch plugin
2.   Github Pull Request Builder plugin

Github Branch source plugin

This is more easy to use and more simple.

Note: you can request new test with adding a comment with the following text: /retest

For security reason this build is triggered only for the contributors who are on a whitelistAs an example for these kind of job, check this one: https://buildsgithub.apache.orgcom/jobelek/loggingargo-log4net/ You can see that there are two tabs. The first includes all the branches and the second the active pull requests.

To work with it you need to commit a Jenkinsfile (we have one since HDDS-965) and the Jenkins can create jobs magically for PRs and/or branches based on patterns.

Every PR will have a separated job and the incremental builds are easily followed.

Github Pull Request Builder plugin

This is a more advanced plugin but not for the multi-branch pipline just for the simple pipeline. It is just one job which can build all PRs (and branches), but there is no separated job instances for the separated PRs.

It's more flexible: we can define github labels to exclude/include and we can define the users.

(see this thread: https://mail-archives.apache.org/mod_mbox/www-builds/201901.mbox/browser if you interested why is it important to trigger build only for the whitelisted users/organizations)

Which approach is the best

...

ozone/blob/master/klepif.yaml It will be modified soon to include this list in the repository (need repo separation)

Reference repositories

1. https://github.com/elek/argo-ozone
   1. the argo (=kubernetes workflow engine) definition
   2. Kubernetes job definition
   3. Configuration of the pull request poller
2. https://gitub.com/elek/klepif
   1. This script polls the gihub api a checks the new commits and comments (handle the /label and /retest commands and submit new argo jobs) prow
3. https://github.com/elek/ozone-ci-1t
   1. Contains all the test results (pr builds and nightly runs)

Future directions

• Github actions are under evaluation. With github action we can remove the requirements of personal repositories
  • the glue code + build definition to execute the checker scripts can be committed to the main repo.
  • We don't need separated repository to store the build data

...

Current problems (Update1)

Summary: The second approach seems to be more better for Ozone (as of now)

1. The two approaches requires different github notifier plugin (I prefer to attach the results of the builds as github 'status' objects). The notifier plugin of the second approach is not yet installed on builds.apache.org
2. There is no clean way to use a service user. The service user needs a pull privilege which can be assigned only to real people according to the apache policies. (See this issue for more information: https://issues.apache.org/jira/browse/INFRA-17449)

Workarounds: as of now the experimental Jenkinsjob uses my credentials and started from a linode cloud server (I got free 50$ at ApacheCon, which is used here to support apache development). I will request the installation of the required notify plugin from INFRA to migrate it to the builds.apache.org. Until that I will try to find a way to publish the admin password (it's an empty box, I have no personal info there, happy to share)

Update2:

1. I don't like the Github Pull request builder any more. It adds random comments to the github if the author is not white listed.I think the right approach is to use something like prow: an external tool which can trigger the build and handle the comment based workflows. I am investigating the possible tools, and in the meantime I use a simple script to trigger the PRs (if they are marked with ozone labels).
2. Ozone checks still run on an experimental location (ci.anzix.net) because I need a lot of experiments and testing and we have not enough resource at builds.apache.org. Jenkinsfile is committed to the hadoop-ozone and the jobs will be migrated t builds.apache.org after stabilization.

Future directions

Kubernetes (and other CNCF projects) use prow to manage the CI workflow.

It's an intelligent (pluggable) github PR bot + kubernetes job scheduler. The bot itself can be managed by /commands which are added to the PRs as comments.

As we have no public k8s server to test Ozone, the prow can't be used for us, but the approach could work. We may need some simple script which can do simple things on the PRs based on the current status or comments:

For example

• Add the ozone label for Ozone only PRs
• Ask the contributor to open a Jira if it's not included in the title
• Close PR if the Jira is closed