...

Release files must be signed with an OpenPGP compatible key. If you do not already have a key for signing Apache releases, follow the developer instructions in the Daffodil KEYS file to generate a key and add it to the KEYS file. Follow the contributor workflow and create a review branch and pull request to commit your changes to the KEYS file. Once merged, perform the following steps:

• Clone the Apache Dist Daffodil release directory, copy the KEYS file, and commit it:

  Code Block
  language bash
  $ svn checkout https://dist.apache.org/repos/dist/release/incubator/daffodil/ daffodil-dist $ cp daffodil.git/KEYS daffodil-dist $ cd daffodil-dist $ svn ci -m "Update Apache Daffodil KEYS"

  
  

• Add your key fingerprint to https://id.apache.org. To get your fingerprint, run the following:

  Code Block
  language bash
  $ gpg --fingerprint KEYID

  
  

• Send your key to a keyserver via the command:

  Code Block
  language bash
  $ gpg --send-keys KEYID

  
  

...

1. Build the Daffodil release candidate image:

   Code Block
   language bash
   $ podman build -t daffodil-release-candidate /path/to/daffodil.git/containers/release-candidate/

   This may take 10-15 minutes the first time, but should be significantly faster in subsequent runs unless something changes in the image where a full rebuild is needed.
   
   

    This requires substantial free disk space.  If you need to control where podman puts this storage, you can specify the --tmpdir, --root, and --runroot directories as ABSOLUTE paths. (Relative paths are known to cause issues.). Note that if you specify these additional flags on this podman build command you must also specify them for the podman run command below.
     

2. Run the daffodil release candidate container. The command is here, but before running it, prepare the necessary inputs (below) that you will need to provide it when prompted. There are timeouts, and if you wait to dig these things up when prompted they will time out and you'll need to start over.  Also, you must run this command in a standard terminal window that supports cursor-positioning via ANSI escape sequences, which it uses heavily.

   Code Block
   language bash
   $ podman run -it --rm \ -v ~/.gitconfig:/root/.gitconfig \ -v ~/.gnupg/:/root/.gnupg/ \ -v ~/.ssh/:/root/.ssh/ \ --hostname daffodil.build \ daffodil-release-candidate

   Note that the -v option is used to bind mount files/directories from the host system in the container so the container has access to git configurations and gpg/ssh keys.

     
     The --dry-run  option can be provided to the very end of the above podman command (e.g. ... daffodil-release-candidate --dry-run ) to perform a test without publishing any artifacts.
   
     The --entrypoint /bin/bash option can be provided before the last argument in the above podman command (e.g. ... --entrypoint /bin/bash daffodil-release-candidate ) to interactively view the configuration settings and manually run the daffodil-release-candidate script. This can be useful for debugging the container or testing changes.
       

3. The container will periodically ask for user input (e.g. usernames, passwords) to sign and publish release files. (Dig out all these before you run the script above so you are ready to provide them when prompted. ) This includes:
     

   1. Release candidate label. For example: rc1 if this is the first release candidate. Note that this should not include the release version number (e.g. 2.0.0-rc1 )–it should only contain the rcX  part. Note that neither this nor any other prompted input has any default value. You must type in the "rc1" (or other) string.
        
   2. Long format of your signing key ID. Available keys will be displayed before the script asks for the keys, but can also be found by running gpg --list-secret-keys --keyid-format long . The key ID should be the 40 character one to minimize chance of collisions. Ensure this is the key ID for your apache.org email for your signing key, not some other identity.
       
   3. Git name and email (e.g. "John Doe" and "john.doe@company.com"). This is the name (first name and surname) and email you want to show up as the "Committer" when the release script creates a git tag or commit, which could potentially be different than what is the the bind mounted .gitconfig  file. This is not your GitHub or Apache credentials--simply the name and email address you use for Daffodil commits.
      
        If you have Github's Email Privacy setting on, be sure to use the private email provided in your Github Settings, otherwise the complete-release script will fail.
       
   4. Apache username and password. This is the username and password credentials used to log in at https://id.apache.org.
       
   5. GitHub SSH key password. The Daffodil repository will be cloned using SSH authentication. If your SSH key is password protected, you may be prompted for that password.
        
   6. Private GPG password (also called the 'pass phrase'). The release process will sign artifacts with your private GPG key–you will be prompted for a password to enable it to use your private key to sign these artifacts.
        

4. After entering the necessary information the script will run. It will output a few error messages that are not true errors about gpg using your private key as the default key for signing, and a few others also.  The script will perform the actions listed below. 

   1. Create a zip of the source
      
      
   2. Create tgz, zip, msi, and rpm of the helper binary
      
      
   3. Create sha512 checksum and ASCII armored detached signatures of the above files
      
      
   4. Place the above files to the Apache dist dev directory. This svn files are not comitted until further action is taken.
      
      
   5. Create javadoc and scaladoc and move to the daffodil site repository docs directory for this release, and create a commit. This commit is not pushed until further action is taken.
      
      
   6. Create a signed git tag. This tag is not pushed until further action is taken.
      
      
   7. Stage jars/poms to https://repository.apache.org
        
5. Once the script completes, you should verify the existence generated files. Note The script will list the files and locations to verify. This includes:
   
   

   1. Verify the checksums and signatures are created in the Apache dist directories and are read for commit, for example:

      Code Block
      $ cd /root/incubator-daffodil-dist/ $ ls -R $ svn status

        

   2. Verify the git tag is attached to the correct commit in the Daffodil repo, for example:

      Code Block
      language bash
      $ git -C /root/incubator-daffodil/ log -n 1

          

   3. Verify the javadoc and scala docs for the version to be released exist in the daffodil site repository.

      This can be done with git log:

      Code Block
      language bash
      $ git -C /root/incubator-daffodil-site/ log -n 1 -p

      or via the file system:

      Code Block
      $ cd /root/incubator-daffodil-site/ $ ls -R

      
      

   4. Verify the all the expected jars/poms at https://repository.apache.org/ exist  (i.e for all modules, both scala versions exists, and for each scala version, there exists a jar, javadoc jar, sources jar, and pom, and for each of those files there exists the armored ASCII file, and the associated md5 and sha1 checksums of all of these).
      
      To do so, visit that url, login in the top right using id.apache.org credentials, select "Staging Repositories" on the left, and find the orgapachedaffodil-XXXX repository. Inspect the "Content" tab to make sure the appropriate jars are uploaded and appear valid.
     
6. If any of the above do not look correct, perform the following steps:
     
   1. Drop the published jars/poms at https://repository.apache.org--check the box for the staging repository just created and choose "Drop" at the top)
       
   2. Type exit  to close the container. All files/commits created in the container will be deleted.
       
   3. Fix the issue and repeat the "Create Release Candidate" process from the beginning.
        
7. After verifying all is correct, follow the instructions to complete the release candidate. These steps include:
      

   1. Run the command:

      Code Block
      language bash
      $ /root/complete-release

      All the previous commands have prepared commits and tags in the three repositories (Apache Dist, Daffodil, and Site). Running this command will push those commits and tag to the remote repositories.
        

   2. Close the staged file at https://repository.apache.org
          
   3. Type exit  to close the container.

...

Create a new release file in the site/_releases/ directory in the Daffodil Site Repository, updating the page to include a summary of the changes and links/descriptions of the bugs that were fixed in this release. The usual procedure for gathering the tickets of interest is as follows:

...

released: false
artifact-root: "https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.0.0-rc1/"
checksum-root: "https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.0.0-rc1/"

Note that these links refer to https://dist.apache.org instead https://downloads.apache.org because this is a pre-release.

Additionally, update the Unsupported Features page if any features/errated are now supported in this released.

...

Subject: [VOTE] Release Apache Daffodil (incubating) 2.0.0-rc1
 
Hi all,

I'd like to call a vote to release Apache Daffodil (incubating) 2.0.0-rc1.

All distribution packages, including signatures, digests, etc. can be
found at:

https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.0.0-rc1/

Staging artifacts can be found at:

https://repository.apache.org/content/repositories/orgapachedaffodil-1000/

This release has been signed with PGP key 36F3494B033AE661, corresponding
to slawrence@apache.org, which is included in the KEYS file here:

https://downloads.apache.org/incubator/daffodil/KEYS

The release candidate has been tagged in git with v2.0.0-rc1.

For reference, here is a list of all closed JIRAs tagged with 2.0.0:

https://s.apache.org/daffodil-issues-2.0.0

For a summary of the changes in this release, see:

https://daffodil.apache.org/releases/2.0.0/

Please review and vote. The vote will be open for at least 72 hours
(Sunday, 11 February 2018, 12 Noon EST).

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

Thanks,
- Steve

...

Subject: [CANCELED][VOTE] Release Apache Daffodil (incubating) 2.0.0-rc1
 
Officially canceling the VOTE for 2.0.0-rc1. I'll create an rc2 and start a new VOTE.

VOTE thread:

https://lists.apache.org/thread.html/ra2fcf855251ed00c354abf29dfce73177fa3bf3a3705c5714f1aaabd%40%3Cdev.daffodil.apache.org%3E

Thanks,
- Steve

...

Subject: [RESULT][VOTE] Release Apache Daffodil (incubating) 2.0.0-rc1
The VOTE to release Apache Daffodil (incubating) 2.0.0-rc1 is now
closed. The vote passes with 5 +1 votes, and NO -1 votes.

The VOTE thread:

https://lists.apache.org/thread.html/c8df54668fbcb7b8285f3e2cc524eac7cb82a721fa823ea5ae7edbe3%40%3Cdev.daffodil.apache.org%3E

+1's were:
  [List of +1 Voters]

Thanks to everyone who voted. I'll open the VOTE thread on the Incubator
general list shortly.!

If the VOTE does not pass, fix the issues, "Drop" the release (i.e closed repo under Staging Repositories) at https://repository.apache.org, repeat the "Create Release Candidate" process from the beginning with a new rc number. Note that creating a new release candidate will automatically delete the previous one.

...

Promoting to Final Release

...

You must now create a VOTE thread on general@incubator.apache.org to get approval from the Apache Incubator Project Management Committee for release.

 Make sure to update alll links and use "plain text" editing for the email. HTML editing often leads to broken links or incorrect formatting.

...

...

The following steps should be taken once the above vote passes.

1. Move the release candidate files to the release directory. (This prompts for your apache account password. It may prompt you about plaintext password storage. If so, it works even if you answer "no".)

   Code Block
   language bash
   $ svn mv -m "Release Apache Daffodil

...

1. 2.0.0

...

1. " \ https://dist.apache.org/repos/dist/dev/daffodil/2.0.0-rc1

...

1. / \

...

...

...

...

1. https://dist.apache.org/repos/dist/release/daffodil/2.0.0/

     

2. Make a new clean clone (for good measure), create a signed git tag based on the release candidate tag. You need your KEYID (See step 3b above) for your apache signing key, and you will be prompted for the pass-phrase for your apache signing key.

   Code Block
   language bash
   $ cd /tmp $ git clone git@github.com:apache/daffodil.git daffodil-2.0.0-rc1 $ cd daffodil-2.0.0-rc1 $ git tag -as -u KEYID -m "Release v2.0.0" v2.0.0 v2.0.0-rc1^{} $ git push origin v2.0.0

   You can delete the /tmp clone directory  at this point.
     

3. Release the published Nexus files by visiting https://repository.apache.org, log in, find the release in "Staging Repositories" and selecting "Release".
   
   
4. Update the JIRA versions to mark the version as released (need to be PPMC).
     

5. Give approximately 24 hours for the release files to sync to mirrors and Maven Central. To verify, check here
     

   1. https://search.maven.org/ (ensure you can see both versions; select one and verify the release versions exist in at least the sbt version)
        

   2. https://www.apache.org/dyn/closer.lua/daffodil/2.6.0 (pick a mirror and ensure the files exist)
       

6.  Once the mirrors have synced, make the following changes to the daffodil site repository and publish them:

     

   1. Modify the release page to have the following parameters:

      Code Block
      released: true date: <date of release> artifact-root: "https://www.apache.org/dyn/closer.lua/daffodil/2.0.0

...

1. 1. /

...

1. 1. " checksum-root: "https://downloads.apache.org/daffodil/2.0.0/"

      
      

   2. Modify the release page of the previous release to use archived root URLs, for example:

      Code Block
      artifact-root: "https://

...

1. 1. archive.apache.org/

...

1. 1. dist/

...

1. 1. daffodil/1.0.0/" checksum-root: "https://archive.apache.org/

...

1. 1. dist/daffodil/

...

1. 1. 1.0.0

...

After at least 72 hours, if the VOTE passes with at least 3 binding +1's, create a RESULT thread announcing the passage.

Code Block

Subject: [RESULT][VOTE] Release Apache Daffodil (incubating) 2.0.0-rc1 The VOTE to release Apache Daffodil (incubating) 3.0.0-rc1 is now closed. The vote passes with 3 +1 (binding) votes and no 0 or -1 votes. The VOTE thread: https://lists.apache.org/thread.html/rbdbc0144c43038ddf0d5b7983e00ffb4ccf013b6121c99a74b0a762c%40%3Cgeneral.incubator.apache.org%3E +1's were: [List of +1 Voters] Thanks to everyone who voted! We will proceed with the official release of 2.0.0.

As before, if the VOTE does not pass, fix the issues, "Drop" the release at https://repository.apache.org, repeat the "Create Release Candidate" process from the beginning with a new rc number. Note that creating a new release candidate will automatically delete the previous one.

Promoting to Final Release

The following steps should be taken once the incubator vote passes.

...

Move the release candidate files to the release directory. (This prompts for your apache account password. It may prompt you about plaintext password storage. If so, it works even if you answer "no".)

$ svn mv -m "Release Apache Daffodil (incubating) 2.0.0" \
    https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.0.0-rc1/ \
    https://dist.apache.org/repos/dist/release/incubator/daffodil/2.0.0/

...

Make a new clean clone (for good measure), create a signed git tag based on the release candidate tag. You need your KEYID (See step 3b above) for your apache signing key, and you will be prompted for the pass-phrase for your apache signing key.

$ cd /tmp
$ git clone git@github.com:apache/incubator-daffodil.git incubator-daffodil-2.0.0-rc1
$ cd incubator-daffodil-2.0.0-rc1
$ git tag -as -u KEYID -m "Release v2.0.0" v2.0.0 v2.0.0-rc1^{}
$ git push origin v2.0.0

You can delete the /tmp clone directory  at this point.
  

...

Give approximately 24 hours for the release files to sync to mirrors and Maven Central. To verify, check here
  

1. https://search.maven.org/ (ensure you can see both versions; select one and verify the release versions exist in at least the sbt version)
     

2. https://www.apache.org/dyn/closer.lua/incubator/daffodil/2.6.0 (pick a mirror and ensure the files exist)
    

 Once the mirrors have synced, make the following changes to the daffodil site repository and publish them:

1. 1. /"

      
      

   2. Modify the site/doap.rdf file to include the release date and version, for example:

      Code Block
      language xml
      <release> <Version> <name>Apache Daffodil</name> <created>2018-02-18</created> <revision>2.0.0</revision> </Version> </release>

      
      

   3. Update the symlink  to the latest Javadoc and Scaladocs in the site/docs directory (note that there is no forward slash at the end of latest).

      Code Block
      language bash
      $ cd site/docs/ $ ln -sfn 2.0.0 latest

      
      

   4. Follow the steps in the README in that repository to test and publish the new release page.
        

2. Now that the download URLS of the previous release point to the archive, remove that release from Apache dist to free up space on mirrors:

   Code Block
   $ svn delete -m "Archive Apache Daffodil 1.0.0" \ https://dist.apache.org/repos/dist/release/daffodil/1.0.0/

     

Announce the release

1. Send an announcement email from your apache.org email address to announce@apache.org, dev@daffodil.apache.org, and users@daffodil.apache.org, (note: send three separate emails instead of one email with multiple TO/CC's), with the below template.
   
     To send to announce@apache.org, your email app needs to be configured. Settings can be found on Apache's Committer Email page. For Gmail, you can request to "Send Email as" and with Thunderbird, you can add a new Outgoing SMTP Server and create a new Identity (Manage Identities) to send from the relay.
   
    Make sure to update links and use "plain text" editing for the email. HTML editing often leads to broken links or incorrect formatting.

   
   

   Code Block
   language text
   Subject: [ANNOUNCE] Apache Daffodil 2.0.0 Released   The Apache Daffodil community is pleased to announce the release of version 2.0.0. Notable changes in this release include <short summary of changes>. Detailed release notes and downloads are available at

Modify the release page to have the following parameters:

released: true
date: <date of release>
artifact-root: "https://www.apache.org/dyn/closer.lua/incubator/daffodil/2.0.0/"
checksum-root: "https://downloads.apache.org/incubator/daffodil/2.0.0/"

Modify the release page of the previous release to use archived root URLs, for example:

artifact-root: "https://archive.apache.org/dist/incubator/daffodil/1.0.0/"
checksum-root: "https://archive.apache.org/dist/incubator/daffodil/1.0.0/"

Modify the site/doap.rdf file to include the release date and version, for example:

<release>
    <Version>
        <name>Apache Daffodil</name>
        <created>2018-02-18</created>
        <revision>2.0.0</revision>
    </Version>
</release>

Update the symlink  to the latest Javadoc and Scaladocs in the site/docs directory (note that there is no forward slash at the end of latest).

$ cd site/docs/
$ ln -sfn 2.0.0 latest

...

Now that the download URLS of the previous release point to the archive, remove that release from Apache dist to free up space on mirrors:

$ svn delete -m "Archive Apache Daffodil (incubating) 1.0.0" \
    https://dist.apache.org/repos/dist/release/incubator/daffodil/1.0.0/

Announce the release

1. Send an announcement email from your apache.org email address to announce@apache.org, dev@daffodil.apache.org, and users@daffodil.apache.org, (note: send three separate emails instead of one email with multiple TO/CC's), with the below template.
     To send to announce@apache.org, your email app needs to be configured. Settings can be found on Apache's Committer Email page. For Gmail, you can request to "Send Email as" and with Thunderbird, you can add a new Outgoing SMTP Server and create a new Identity (Manage Identities) to send from the relay.
    Make sure to update links and use "plain text" editing for the email. HTML editing often leads to broken links or incorrect formatting.

   Code Block
   language text
   Subject: [ANNOUNCE] Apache Daffodil (incubating) 2.0.0 Released   The Apache Daffodil (incubating) community is pleased to announce the release of version 2.0.0. Notable changes in this release include <short summary of changes>. Detailed release notes and downloads are available at: https://daffodil.apache.org/releases/2.0.0/ Daffodil is an open source implementation of the Data Format Description Language (DFDL) specification that uses DFDL schemas to parse fixed format data into an infoset, which is most commonly represented as either XML or JSON. This allows the use of well-established XML or JSON technologies and libraries to consume, inspect, and manipulate fixed format data in existing solutions. Daffodil is also capable of the reverse by serializing or "unparsing" an XML or JSON infoset back to the original data format. For more information about Daffodil visit: https://daffodil.apache.org/ Regards, The Apache Daffodil Team ===== *Disclaimer* Apache releases/2.0.0/ Daffodil is an effortopen source undergoingimplementation incubationof atthe TheData ApacheFormat SoftwareDescription FoundationLanguage (ASFDFDL), sponsoredspecification bythat theuses ApacheDFDL Incubatorschemas PMC. Incubationto parse fixed format data into an infoset, which is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.most commonly represented as either XML or JSON. This allows the use of well-established XML or JSON technologies and libraries to consume, inspect, and manipulate fixed format data in existing solutions. Daffodil is also capable of the reverse by serializing or "unparsing" an XML or JSON infoset back to the original data format. For more information about Daffodil visit: https://daffodil.apache.org/ Regards, The Apache Daffodil Team

   
   

2. The same or a similar announcement can also be posted to the DFDL Workgroup News page here: https://redmine.ogf.org/projects/dfdl-wg/news  (Done by a PPMC member, like Mike B).
     

3. Send a tweet from the @ApacheDaffodil twitter account, mentioning the release version, highlights of changes, and a link to the release page. You will need to be invited/accept, via tweedeck,  permission to tweet from the account. At which point you may compose the below style of message from Tweetdeck.

   Code Block
   The @ApacheDaffodil team is excited to announce the release of version 2.4.0! Notable changes include <summary of changes>. Full details and downloads at https://daffodil.apache.org/releases/2.4.0/

   
   

4. Various published DFDL schemas and examples will, most-likely, want to be updated so that they automatically reference the latest release of Daffodil from their master branch. (Owl/Tresys people can use the utils/set-daffodil-version in the Gitea dfdl-schemas module which has subprojects for every DFDL schema project we know about, public or private.)

...