[DRAFT] How to Become a CloudStack Release Manager

Note: this is a draft written by individual(s) and no way represents the guidance or stand of the CloudStack PMC.

Anybody who is acknowledged by the Apache CloudStack community and PMC, may become a release manager, however, the standard practice is that usually, a release manager needs to be at least a committer.

References/reading: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Releases

First Steps

Glad you want to become a release manager, here’s what you need to do:

Volunteer and express interest that you want to be RM
Build support with community and ensure that you actually have time for RM work (for example get support from your work colleagues and employer) which could even last a quarter or more.
Propose yourself as a RM on dev+users mailing list
Discuss with the community which features MUST be included in the release and plan a timeline to ensure the required features are included
Propose the timeline to the dev+users@ mailing list. If you're not certain about dates, avoid exact dates on the proposed timeline but give some idea (such as end of month XYZ?)

Suggested Checklist for RMs and co-RMs

Create milestone for the release on Github
Triage all open issues and PRs, with labels and severity; with the community
Estimate the release timeline and resources necessary based on triaged list of issues
Our experience shows, a single dev. can target 3-5 issues per week andabout 5-10 PRs per week
Create a health check PR and invoke test matrix at least weekly - this helps in case of any regression to find the latest successful run and ensure the integrity of the main branch
Before merging a PR verify the following:
- At least 2 LGTM/approvals, one for code review and another for manual testing
- Successful test results (doesn’t mean 0 failing tests 100% of the times, as there could be the case that there are known issues - which should be in progress of fixing)
Communicate regularly with the community
Ask/discuss for a QA/test plan to execute before cutting the RC; at least ensure health check PR is passing and there are no obvious regressions
Ask community for help as needed

Learning by Example

Let’s take the recent (at the time of writing this) 4.15.1.0 to learn release management.

The target release, 4.15.1.0, milestone issues and PRs are tracked under:
https://github.com/apache/cloudstack/milestone/17

The release management follows the process as per:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+Procedure

Here’s what proposing yourself as a RM may look like: (on dev+users list)

https://markmail.org/message/xyuir55nppdvr4fq

Here’s what cutting a RC and starting vote looks like: (on dev+users list)

https://markmail.org/message/lactwcivi65z7zov

For building convenience packages, ask community who have access to Jenkins to build the packages and sync from the staging server to the community server: http://download.cloudstack.org; additional ask/get access to CI/CD (for example to run pkging and smoketests using the @blueorangutan bot)

NOTE: Before generating an RC ensure your locale is set to an English variation. For example: en_US.utf-8

Here’s what the final RC passing vote looks like: (on dev+users list)

https://markmail.org/message/im5jfhvkbxwj5suu

Here’s what working on release announcement/draft looks like (on marketing list):

https://markmail.org/message/bjtoqk2oprxng6fa

Here’s what announcing the release looks like:
https://markmail.org/message/2ypynih3li7bllrl

Appendix: Additional steps on the Release Procedure

1) Set next version

After voting is concluded and the RC is merged into the codebase, then the next version has to be set, by invoking the set_next_version script. After executing this script, check the following files (most likely will need a manual fix):

debian/changelog
tools/checkstyle/pom.xml
tools/docker/Dockerfile
tools/docker/Dockerfile.marvin
tools/marvin/setup.py

Do git grep OLD_VERSION to find any remaining occurrence of version not changed.

2) Build packages

Shapeblue Jenkins: build centos7, centos8, debian and suse15 packages on each step (using the RC commit SHA) and

For download.cloudstack.org:
- Build using build ID = “1” (in Jenkins)
- ssh into cloudstack@download.cloudstack.org, or via staging server (10.0.3.120):
  - scp the rpm-only pkgs to relevant folders, and rebuild repo metadata, for example:
  cloudstack@cloudstack:~/repository/centos/8/4.15$ createrepo --baseurl http://download.cloudstack.org/centos/8/4.15 .
  - For deb pkgs, ask Wido/Daan/Rohit to do the pkg builds. In case deb packages not created by other community members, the same can be built using jenkins with build_id=1.
    - create an appropriate destribution directory. For instance
```
$ mkdir -p repository/ubuntu/dists/jammy/4.18/
$ cd repository/ubuntu/dists/jammy/4.18/
$ mkdir binary-all
$ mkdir binary-amd64
$ mkdir pool
```
    - Copy the *.deb files in the appropriate debian/ubuntu distro folders on the http://download.cloudstack.org server. The *.deb files should go in the pool directory.
    - Execute the indexing script for all distributions. I.E. focal, jammy, etc but also binary-all and binary-amd64 and/or binary-i386
```
$ apt-ftparchive -c /home/cloudstack/mirrorscripts/ubuntu/apt-ftparchive/jammy.conf packages /srv/mirror/cloudstack/ubuntu/dists/jammy/4.18/pool/ > /srv/mirror/cloudstack/ubuntu/dists/jammy/4.18/binary-all/Packages
$ apt-ftparchive -c /home/cloudstack/mirrorscripts/ubuntu/apt-ftparchive/jammy.conf packages /srv/mirror/cloudstack/ubuntu/dists/jammy/4.18/pool/ > /srv/mirror/cloudstack/ubuntu/dists/jammy/4.18/binary-amd64/Packages 
```
    - Execute the indexing script
```
`/home/cloudstack/mirrorscripts/index-ubuntu-archive.sh`
```
    - When replacing packages repository/ubuntu/.cache/packages-x.y-all.db may need to be deleted.
    - finally a test script can be executed:
```
bash -x ~/mirrorscripts/testubu.sh /srv/mirror/cloudstack/ubuntu/
```
For packages.shapeblue.com:
- Ask Daan/Rohit to assist

3) ACS Docs for release notes:

After merging the documentation PR, for publishing changes head over to: https://readthedocs.org/projects/cloudstack-documentation/

Go to Admin -> edit versions, or

https://readthedocs.org/projects/cloudstack-documentation/versions/

Activate the new tag (for ex. 4.15.2.0)
Admin -> adv settings -> select/update default branch to 4.15.2.0 (new tag) which is what "latest" points to
Check or kick builds for the new tag and latest;

https://readthedocs.org/projects/cloudstack-documentation/builds/

4) ACS Project website

Make changes to the release data in this repo: https://github.com/apache/cloudstack-www
Ask Daan/Rohit/Others PMCs and community to help
Currently, the website is published via the asf-site branch which could change in future
For major release, you'll need to build and add the API docs to cloudstack-www as well

5) Github Release

Once you've the doc links live, create a Github release; for example:

https://github.com/apache/cloudstack/releases/tag/4.15.2.0

6) Announcement & Blog

Send draft to marketing@ first, when approved send to all MLs (individually): use your Apache email account and send the mail on plain text and use "https" links wherever possible

For announcement email, see example: https://markmail.org/thread/iemn7p7d62avhfrp

After sending email, copy/paste and publish the announcement on ACS blog: https://blogs.apache.org/cloudstack/

7) Update LTS timeline on cwiki

Update Release calendar section here, https://cwiki.apache.org/confluence/display/CLOUDSTACK/LTS

Space shortcuts

Child pages