Development Process

This document serves as the touchstone for the development process that is used by the Maven project to manage branches and the trunk during the development leading up to releases.
In particular the following will be addressed:

• New development
  • Where new development starts
• Releases
  • Creating a release branch
  • Working on a release branch
  • Creating release candidates (RCs) from a release branch
  • Soak period for RCs
  • How to integrate bug fixes into an RC
  • Generating an official release
  • How to integrate bug fixes into a release branch
• Experimental, complex bug fix, and high-risk change branches

The terminology used here is somewhat SVN specific as that is the SCM the Maven project uses. At some point we would like to generalize these strategies for all SCMs and integrate these strategies into general tools where Maven SCM can deal with the operations outlined here in an SCM independent fashion.

New development

How new development starts

The discussion of what to resolve will happen on the list or in irc but then get populated in JIRA so that what's up for discussion gets captured. JIRA can ping the list with unresolved issues and as we are discussing the issues I think someone needs to act as secretary and capture the salient ideas in the wiki. So for the dev process stuff I created a document and steward that document to completion. I think the same would go with the integration tests where this is something that Vincent is keen to resolve so he can be the secretary for that issue (if that's ok Vincent). So I think the flow becomes

• everyone pushes issues they want resolved into JIRA
• we prioritize a short list to work on in a particular week (or whatever), we can use votes or just decide amongst ourselves

with that short list we:

• pick the issue at hand
• burst of discussion on the list
• secretary captures the salient points
• offers up the document for review
• go back to discussion/capture/review until complete
• a final document then accompanies the resolution and the issue is closed

NOTES:

• a filter can be created to send all unresolved issues to the dev list
• JIRA is used as a tracker not a place for discussion
• TODO do we want a separate project for design/best practices discussions
• possibility of setting up a custom workflow
• demonstrate design/practices as opposed to scheduled work
• tagging issues with multiple components so that an issue is marked with its real category but can also be marked with a meta category like "design" or "best practices" which would allow us to group them in a view and be able to report on them
• need to sync all the bits and pieces in jira and confluence so we have a cohesive view of the work that needs to be done and planned for

One person might start a discussion but it can be picked up by anyone who has the energy/motivation/time/whatever to finish it. I think what happened with the dev process works just fine. The issue was in JIRA with me assigned, but you had time to post some initial thoughts and I tried to follow up with a document. The issue needs a champion but anyone should be able to carry it to completion because all the information should be clearly visible.

once its in JIRA, it doesn't need a
champion. I was just making sure it didn't get buried, but
responsibility for momentum can be carried by anyone able to at any
given time.

Where new development starts

All new development, whether new features or general improvements, should be integrated into the latest revision or the trunk. Once a new feature is implemented and integrated into the trunk it can then be decided which branches the new development should be applied to. There may be rare cases where a new development only applies to a particular branch but in general all new features go to the trunk first and disseminated from there. The important thing is that all changes go into the trunk first, then get merged into the release branch as needed, so no changes ever get dropped by mistake. Changes should be kept in sync at all times where possible and this is the responsbility of the release manager. So developers can focus on improvements in the trunk and the burden of merging will be the task of the release manager. New features or improvements should never originate from a branch

The responsibility of merging changes from the trunk to the various branches is the responsbility of the release manager for the branch in question. To be clear if you are committing changes to trunk do not merge your changes into any of the release branches.

Releases

Creating a release branch

• A release branch should be made well in advance of the release to allow for stabilzation of the release and the preparation of RCs. A release branch is a long-lived branch from which all the releases for a non-API breaking versions are made. So you would release 2.0, 2.0.1, 2.0.2 of a project from a 2.0.x branch and likewise you would release 2.1, 2.1.1, 2.1.2 from a 2.1.x branch.
• Make sure that all developers checked in their local modifications before the branch is created.
• Copy using URLs as everything will occur on the server and be faster.

To create a release branch in preparation for a release use the following command:

svn copy -m "Create Maven 2.0.x branch" \
svn://svn.apache.org/repos/asf/maven/components/trunk \
svn://svn.apache.org/repos/asf/maven/components/branches/maven-2.0.x

Working on a release branch

To work on a release branch you can either checkout the branch or switch and existing working copy to the branch. It is probably easier to checkout the branch in a different directory so that you can work on the trunk and branch at the same time.

svn co svn://svn.apache.org/repos/asf/maven/components/branches/maven-2.0.x

Calling a vote for a release

• A developer posts a message stating it's time for a release for a given component. If there is a lead for the given component then the lead should make the post or the lead should be consulted before the post to prevent any potential confusion.
• At this point no more issues can be assigned a fix version corresponding to the release
• When the count goes to zero the voting for the release starts
• 72 hours later the release occurs, or we discuss why the release was voted down

At some point we might even be able to automate this will a little voter app, or use the one we use for board elections. Still visible but much better audit trail.

Creating release candidates (RCs) from a release branch

RCs should be made available in succession until the community is satisfied that the RC in question is of release quality. The RC that finally makes the cut as the release should be used as it was originally built so the RCs will be named as if they were the final release. This means that we have a few technical issues to resolve:

• TODO We need a staging artifact repository where the RCs can be placed so that failed RC attempts don't pollute a release artifact repository
• TODO We need a reliable way of moving the successful RC from the staging repository to the release repository. This is an intended feature for the Repository Manager but we may need a stop-gap solution until the Repository Manager is ready for production use.
• TODO We need to ensure that the RC that gets promoted as the release is not rebuilt. At the same time we need to provide a way to identify what version a user is running (be it a RC1, RC2, etc). One solution is to use a build# so that a maven2 version is always made of both a Version + a build number. For example you would have "Maven 2.1 Build # 1353" which would be RC1 and then "Maven 2.1 Build # 1450" which would be RC2 which gets promoted as the 2.1 release.

Soak period for RCs

RCs should be circulated for no less then three days so that we can accurately determine if there are any defects present. When you are ready to release the RC use the release plug-in:

mvn release:prepare
...
mvn release:perform
...

• TODO We need to figure out the process of how we tag the RCs, probably don't need to keep them. Maybe just roll over the previous one until the RC is good enough to release.

How to integrate bug fixes into an RC

If bugs are found in the RCs, then the fixes should be applied to the trunk and then it will be up to the release manager to integrate the fixes into the release branch.

• TODO Some tag in the SCM message might be useful to give a hint to the release manager, or a tool, as to which revisions need to be merged.

Generating an official release

Release external dependency snapshots

For each:

• Call a vote 72 hrs. ahead
• Release the library using the release plugin
• Update any site documentation referring to the released version
• Announce the release

Call the vote

• Alllow at least 72 hrs.
• Work should be very near to completion when vote is called.

Release the libraries

• Resolve any -SNAPSHOT dependencies that are external to Maven's multimodule build.

How to integrate bug fixes into a release branch

If bugs are found in the release then the fixes should be applied to trunk and then it will be up to the release manager to integrate the fixes into the release branch. The only time this would no apply is when there are features in the branch that are not present in the trunk i.e. deprecated features.

• moving into a API breaking version there will be alphas, betas i.e. 2.1-alpha-1
• we need to set up multiple CI processes
• possibility of a merge file

Experimental, complex bug fix, and high-risk change branches

TODO

The use of the svnmerge script to help pick off the revisions to merge.

The bottom of this link some example usage of the svnmerge.py script
http://gcc.gnu.org/wiki/SvnBranch

The svnmerge script is a wonderful little tool. In a nutshell, you can take any branch and 'svnmerge init' it against any other branch (or the trunk by default) and then by typing 'svnmerge avail' obtain a list of changesets that are available to merge over from that branch/trunk to your working copy. 'svnmerge merge -rN will merge a particular version over and let you commit it to the branch. Once committed, the svnmerge avail command will no longer list that changeset as available to merge over. This can work either in either a star pattern where each branch can merge changes over from the trunk selectively, or in a chained pattern where branches follow a release process of development->staging->production. In the chained approach checking the available patchs will only give you the changeset available from the source branch allowing for a clean structured code tagging/release process.

You can find the svnmerge.py script here: http://svn.collab.net/viewvc/svn/trunk/contrib/client-side/

Updating the svnmerge-integrated property

If you know that a particular revision is already in sync between the trunk and the branch in question then you need to do the following so that the svnmerge script will ignore the revision when compiling a list of candidate revisions:

cd <the-branch>
svn propedit svnmerge-integrated
<start-edit>
/maven/components/trunk:1-368287,368989,369304,<your-revision>
<end-edit>
svn commit

Keep in mind the revisions are revisions on the trunk. So if you started in the branch and merged to the trunk then make sure it's the id of the revision on the trunk!

Useful links

http://www.asterisk.org/developers/svn-branching-merging
http://www.reactos.org/wiki/index.php/Best_practices_for_working_with_branches
http://svn.apache.org/repos/asf/httpd/httpd/trunk/VERSIONING
http://svn.collab.net/viewcvs/svn/trunk/contrib/client-side/
http://www.freebsd.org/doc/en_US.ISO8859-1/articles/releng/index.html

NOTES:

• a filter can be created to send all unresolved issues to the dev list
• JIRA is used as a tracker not a place for discussion
• TODO do we want a separate project for design/best practices discussions
• possibility of setting up a custom workflow
• demonstrate design/practices as opposed to scheduled work
• tagging issues with multiple components so that an issue is marked with its real category but can also be marked with a meta category like "design" or "best practices" which would allow us to group them in a view and be able to report on them
• need to sync all the bits and pieces in jira and confluence so we have a cohesive view of the work that needs to be done and planned for
• all snapshots should be resolved before the final testing is done on what will ship