Below are the steps one should follow to create a new Apache Daffodil release.

Create a [DISCUSS] thread on dev@daffodil.apache.org to make a decision as a community if the timing is correct for a release and what open issues should be resolved for a release.
Upon agreement, someone should volunteer to be the "Release Manager" to take the responsibility to prepare the release candidate.

Initial Setup

The following steps must only be performed once to setup signing keys and the file distribution SVN repository.

Apache Dist Repository

The Apache dist repository is where files are staged prior to release, and stored once the release has been approved. Follow these steps to create a local checkout of the Daffodil dist directories:

svn checkout https://dist.apache.org/repos/dist apache-dist --depth empty
cd apache-dist/
svn update --set-depth empty dev
svn update --set-depth empty dev/incubator
svn update --set-depth infinity dev/incubator/daffodil
svn update --set-depth empty release
svn update --set-depth empty release/incubator
svn update --set-depth infinity release/incubator/daffodil

Staged files are created in the dev directory and are moved to the release directory once approved by the Apache Incubator Project Management Committee.

Daffodil Site Repository

The Daffodil release script modifies files in the Daffodil website repository, and you will need to make manual changes to the repsository as well. Clone the git repo with the following command:

$ git clone https://github.com/apache/incubator-daffodil-site

Signing Keys

Release files must be signed with an OpenPGP compatible key. If you do not already have a key for signing Apache releases, follow the developer instructions in the KEYS file in the Daffodil repository to generate a key and add it to the KEYS file. Follow the contributor workflow and create a review branch and pull request to commit your changes to the KEYS file. Once merged, perform the following steps:

Copy the KEY file to release/incubator/daffodil/KEYS in the Apache Dist repo and commit it
Add your key fingerprint to https://id.apache.org. To get your fingerprint, run the following:
```
$ gpg --fingerprint KEYID
```
Send your key to a keyserver via the command:
```
$ gpg --send-keys KEYID
```

For more information on signing keys, visit How to OpenPGP and Signing Releases.

SBT PGP

The sbt-pgp plugin is required to publish signed releases. Add the following to the file ~/.sbt/1.0/plugins/pgp.sbt to enable usage of the plugin:

addSbtPlugin("com.jsuereth" % "sbt-pgp" % "1.1.1")

In most cases, no other configuration should be necessary.

WiX Toolset

The sbt-native-packager plugin is used by Daffodil to generate a Windows MSI installer. The plugin depends on WiX Toolset to do the actual MSI generation, which unfortunately only works on Windows. However, the release script must be run on Linux. Below are the steps necessary to configure a Linux system to use the WiX Toolset from the plugin using wine.

Install wine and winetools for you system, for example on Fedora, run:
```
$ dnf install wine winetools
```
Run the command winecfg, and select Windows 10 as the Windows Version.
Install dotnet45 into wine using winetools
```
$ winetricks dotnet45
```
Download wix311-binaries.zip from WiX Toolset version v3.11.1. Note that the wix311.exe setup installer does not work on wine, so it is important to download the precompiled binaries.

Extract that zip into a WIX specific directory:

$ mkdir ~/wix311/
$ unzip wix311-binaries.zip -d ~/wix311/

Run the following command to set the WIX environment variable to that directory. It is reccommended that this also be added to ~/.bashrc:
```
$ export WIX=~/wix311/
```
Add symbolic links to the the WIX directory that point to a custom script in the Daffodil repository:
```
$ ln -s ~/incubator-daffodil/scripts/wix_wine.sh $WIX/\\bin\\candle.exe
$ ln -s ~/incubator-daffodil/scripts/wix_wine.sh $WIX/\\bin\\light.exe
```
Note that the bin, double-slashes, and .exe are necessary, as they work around assumptions made the the plugin about running on a Windows environment.

Creating a Release Candidate

It is highly recommended that you create a new clone of the incubator-daffodil repository in the /tmp directory and perform the following steps in that clone. This is for two main reasons:

This ensures that there are no stray files or commits in your deelopment repository that might lead to reproducability issues.
Generating the Windows MSI installer can fail with "File Not Found" errors if file paths are too long. Cloning the repository into /tmp shortens the root path and reduces the chance of creating long paths that wine/Windows cannot support.

Prior to creating the release candidate, the version setting in build.sbt should contain the -SNAPSHOT keyword. Create and merge a pull request to remove this keyword in preparation for a non-snapshot release.
.
From within the root of the Daffodil directory, execute the scripts/release-candidate.sh script, providing the release candidate label (e.g. rc1), the path to the root of apache-dist directory created above, that path the the root of the daffodil site repository, your Apache login credentials (e.g. for https://id.apache.org), and your long format gpg key id (e.g. gpg --list-keys --keyid-format long KEYID). Run the script like so:
```
$ ./scripts/release-candidate.sh
```
The credentials are used to publish the jars to the Apache staging repo. You will also be asked to enter the password for your private gpg key created above to sign the published jars and the zip/tars . This script will perform the following actions:
1. Create a zip of the source
2. Create tgz, zip, msi, and rpm of the helper binary
3. Create sha256 and sha512 checksum and ASCII armored detached signatures of the above files
4. Move the above files to the Apache dist dev directory
5. Create javadoc and scaladoc and move to the daffodil site repository docs directory for this release
6. Stages jars/poms to https://repository.apache.org
7. Create a signed git tag
Once the script completes, you should verify all the files. This includes:
1. Verify the checksums and signatures created in the Apache dist directories are correct
2. Verify the staged jars/poms at https://repository.apache.org/ are correct. To do so, visit that url, login in the top right using id.apache.org credentials, select "Staging Repositories" on the left, and find the orgapachedaffodil-XXXX repository. Inspect the "Content" tab to make sure the appropriate jars are uploaded and appear valid.
3. Verify the git tag is correct
4. Verify the javadoc and scala docs in the daffodil site repository are correct
If any of the above do not look correct you will need to rerun the release candidate script; however, before doing so you must
1. Delete the files in the Apache dist directory
2. Drop the published jars/poms (at https://repository.apache.org - check the box for the staging repository just created and choose "Drop" at the top)
3. Delete the git tag (git tag -d tagname).
4. Delete the files in the daffodil site repository
5. Fix whatever the issue is and repeat from step 2.
After verifying all is correct, commit the changes:
1. Commit the files in Apache dist, for example:
```
svn add dev/incubator/daffodil/*
svn ci -m "Stage Apache Daffodil (incubating) 2.0.0-rc1"
```
2. Close the published Nexus files by visiting https://repository.apache.org, log in, find the release in "Staging Repositories" and select "Close".
3. Push the git tag
```
git push asf v2.0.0-rc1
```

Update the Website

Create a new release file in the site/_releases/ directory in the Daffodil Site Repository, updating the page to include a summary of the changes and links/descriptions of the bugs that were fixed in this release. Parameters that must be set because this is a release candidate include:

released: false
artifact-root: "https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.0.0-rc1/"
checksum-root: "https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.0.0-rc1/"

Follow the steps in the README in that repository to publish the new release page.

Daffodil Release Vote

With the release files published for staging and a website created, you may now start a vote on these files. To do so, send an email to dev@daffodil.apache.org based on the following example, making sure to update all links and version numbers:

Subject: [VOTE] Release Apache Daffodil (incubating) 2.0.0-rc1
 
Hi all,

I'd like to call a vote to release Apache Daffodil (incubating) 2.0.0-rc1.

All distribution packages, including signatures, digests, etc. can be
found at:

https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.0.0-rc1/

Staging artifacts can be found at:

https://repository.apache.org/content/repositories/orgapachedaffodil-1000/

This release has been signed with PGP key 033AE661, corresponding to
slawrence@apache.org, which is included in the repository's KEYS file.
This key can be found on keyservers, such as:

http://pgp.mit.edu/pks/lookup?op=get&search=0x033AE661

It is also listed here:

https://people.apache.org/keys/committer/slawrence.asc

The release candidate has been tagged in git with v2.0.0-rc1.

For reference, here is a list of all closed JIRAs tagged with 2.0.0:

https://issues.apache.org/jira/browse/DAFFODIL-1864?jql=project%20%3D%20DAFFODIL%20AND%20fixVersion%20%3D%202.1.0%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC

For a summary of the changes in this release, see:

https://daffodil.apache.org/releases/2.0.0/

Please review and vote. The vote will be open for at least 72 hours
(ends on Sunday, 11 February 2018, 12 Noon EST).

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

Thanks,
- Steve

After at least 72 hours, if the VOTE passes, create a RESULT thread announcing the passage.

If the VOTE does not pass, "Drop" the release at https://repository.apache.org, and delete the dist files, e.g.:

svn rm dev/incubator/daffodil/*
svn ci -m "Drop Apache Daffodil (incubating) 2.0.0-rc1"

Once the issues are resolved, start again at Creating a Release Candidate to create a new release candidate with a bumped rc number (e.g. rc2).

Incubator Release Vote

You must now create a VOTE thread on general@incubator.apache.org to get approval from the Apache Incubator Project Management Committee for release. This email should be the same as the above, but with the following at the beginning of the email, again making sure to update the links to the vote and result thread.

Subject: [VOTE] Release Apache Daffodil (incubating) 2.0.0-rc1
 
The Apache Daffodil community has voted and approved the proposed
release of Apache Daffodil (incubating) 2.0.0-rc1.

We now kindly request the Incubator PMC members review and vote on this
incubator release.

Daffodil is an open source implementation of the DFDL specification that
uses DFDL schemas to parse fixed format data into an infoset, which is
most commonly represented as either XML or JSON. This allows the use of
well-established XML or JSON technologies and libraries to consume,
inspect, and manipulate fixed format data in existing solutions.
Daffodil is also capable of the reverse by serializing or "unparsing" an
XML or JSON infoset back to the original data format.

Vote thread:
https://lists.apache.org/thread.html/4b71db31a6a420098a18139a046c5493d5685137251b4727736a9f18@%3Cdev.daffodil.apache.org%3E

Result thread:
https://lists.apache.org/thread.html/f883421a96deffee80e59bd2fbbf07062dfe0ee26e4c4c4cfa194ba5@%3Cdev.daffodil.apache.org%3E

After at least 72 hours, if the VOTE passes with at least 3 binding +1's, create a RESULT thread announcing the passage.

If it does not pass, follow the same steps as if the Daffodil Release Vote fails and create a new release as before.

Promoting to Final Release

The following steps should be taken once the incubator vote passes.

In the Apache dist directory, move the release candidate files to the release directory:

svn mv dev/incubator/daffodil/2.0.0-rc1/ release/incubator/daffodil/2.0.0/
svn rm release/incubator/daffodil/1.0.0/
svn ci -m "Release Apache Daffodil (incubating) 2.0.0"

In the Daffodil git repository, create a signed git tag based on the release candidate tag
```
git tag -as -u KEYID -m "Release v2.0.0" rel/v2.0.0 v2.0.0-rc1
git push asf rel/v2.0.0
```
Release the published Nexus files by visiting https://repository.apache.org, log in, find the release in "Staging Repositories" and selecting "Release".
Give approximately 24 hours for the release files to sync to mirrors and maven central.

Once the mirrors have synced, make the following changes to the daffodil site repository to make the release available:

Modify the release page to have the following parameters:

released: true
date: date of release
artifact-root: "http://www.apache.org/dyn/closer.lua/incubator/daffodil/2.0.0/"
checksum-root: "http://www.apache.org/dist/incubator/daffodil/2.0.0/"

Modify the release page of the previous release to use archived root URLs, for example:

artifact-root: "http://archive.apache.org/dist/incubator/daffodil/1.0.0/"
checksum-root: "http://archive.apache.org/dist/incubator/daffodil/1.0.0/"

Modify the doap.rdf file to include the release date and version, for example:

<release>
    <Version>
        <name>Apache Daffodil</name>
        <created>2018-02-18</created>
        <revision>2.0.0</revision>
    </Version>
</release>

Update the symlink to the latest Javadoc and Scaladocs
```
ln -sfn 2.0.0 site/docs/latest
```
Remove the previous release from the Apache dist repository now that the download URLs point to the archives:
```
svn rm release/incubator/daffodil/1.0.0/
svn ci -m "Archive Apache Daffodil (incubating) 1.0.0"
```

Announce the release

Send an announcement email to announce@apache.org, dev@daffodil.apache.org, and users@daffodil.apache.org, (note: send three separate emails instead of one email with multiple TO/CC's), with the following template:

Subject: [ANNOUNCE] Apache Daffodil (incubating) 2.0.0
 
The Apache Daffodil (incubating) community is pleased to announce the
release of version 2.0.0.

Release notes and downloads are available at:

https://daffodil.apache.org/releases/2.0.0/

Daffodil is an open source implementation of the DFDL (Data Format
Description Language) specification that uses DFDL schemas to parse
fixed format data into an infoset, which is most commonly represented as
either XML or JSON. This allows the use of well-established XML or JSON
technologies and libraries to consume, inspect, and manipulate fixed
format data in existing solutions. Daffodil is also capable of the
reverse by serializing or "unparsing" an XML or JSON infoset back to the
original data format.

For more information about Daffodil visit:

https://daffodil.apache.org/

Regards,
The Apache Daffodil Team

The same or a similar announcement can also be posted to the DFDL Workgroup News page here: https://redmine.ogf.org/projects/dfdl-wg/news
Send a tweet from the @ApacheDaffodil twitter account, mentioning the release version, highlights of changes, and a link to the release page.
Various published DFDL schemas and examples will, most-likely, want to be updated so that they automatically reference the latest release of Daffodil from their master branch.

Page tree

Release Workflow