Tika release process

Before the release

Run mvn ossindex:audit -Dossindex.fail=trueto see if there are any vulnerable dependencies.

Run Consider running mvn versions:display-plugin-updates and mvn versions:display-dependency-updates to identify any updates that need to be made.

Make sure to run full regression tests after making updates with mvn -Prelease-profile clean verify.

Making a release

If you've done this before, have all programs installed, all necessary karma granted, and everything configured, steps 1 though 9 will take around an hour (tests take a while to run). If you've never done this before, you will probably run into an issue not covered in this set of instructions. So, it will take longer (maybe a couple hours, maybe a couple days).

Either way, please update these instructions with any clarifications you think might help the next release manager.

1. git clone https://github.com/apache/tika.git release OR git clone https://github.com/apache/tika.git -b branch_1x

2. Add to the <scm> section in tika-parent's pom

   No Format
   <connection>scm:git:https://github.com/apache/</connection> <developerConnection>scm:git:https://github.com/apache/</developerConnection> <url>https://github.com/apache/tika</url>

   Background: for some reason in the 2.x branch these lines are deleted during each release process.  They are not deleted in the 1.x branch.

3. Update CHANGES.txt with release date (Release X.Y.Z - MM/dd/yyyy (date format matters!) and (if needed) add additional changelog entries.
4. On the Tika JIRA, create versions X.Y.Z, X.(Y+1), and X.(Y+2), if not already done.
   1. Move any lingering unresolved issues from X.Y.Z to X.(Y+1).
      1. Query open issues: e.g. "project = Tika AND resolution = unresolved AND fixVersion = 1.16"
      2. Upper right, select bulk change
      3. Select all, modify to fixVersion = X.(Y+1)
      4. At the bottom of the screen, unselect the email updates
5. git add CHANGES.txt && git commit -m "Update CHANGES.txt for X.Y.Z release." && git push origin
6. Run apachemvn apache-rat:check and fix any missing license headers / add excluded for properly non-licensed fails
7. setenv MAVEN_OPTS "-Xms128m -Xmx256m" (or export MAVEN_OPTS="-Xms128m -Xmx256m" if  if you are using bash)
8. mvn release:prepare
   
   1. Make sure all the release numbers are set to X.Y.Z (from X.Y.Z-SNAPSHOT)
   2. When prompted, the scm tag name should be X.Y.Z (version number). If this is a release candidate, the tag should be X.Y.Z-rcN.
   3. Watch for a BUILD SUCCESSFUL message
   4. Make sure you have gnupg (or variant) installed (see http https://www.apache.org/dev/openpgp.html)
9. mvn release:perform
   1. Make sure you can log on to https://repository.apache.org/
   2. Make sure you have the following entry in your ~/.m2/settings.xml file: http https://maven.apache.org/settings.html#Servers. The server ID should be apache.releases.https.
10. Head over to https://repository.apache.org/ (Apache's Nexus Server)
    1. Login with your ASF username and password
    2. Make sure there is only one repo from your build and that it contains everything. A transmission failure can lead to multiple repositories, none of which are valid... even when mvn release:perform alleges  alleges SUCCESS
    3. "Close" the staging repository with message Apache Tika X.Y.Z release candidate #N.
    4. Copy the URL from the closed staged repository for Tika (you'll need this later)
11. Update CHANGES.txt with a "Release X.Y.Z + 1 - Current Development" section.
12. Check the contents of target/checkout/target/X.Y.Z/* into  into https://dist.apache.org/repos/dist/dev/tika/ (tika-X.Y.Z-src.zip{.asc|.md5|.shasha512}, tika-app-X.Y.Z.jar{.asc|.md5|.shasha512} and tika-server-X.Y.Z.jar{.asc|.md5|.shasha512}, and CHANGES.txt.
    1. Wiki Markup{{svn co https://dist.apache.org/repos/dist/dev/tika/ dist.dev.tika}} 2. {{
    2. cd cd dist.dev.tika}} 3. {{svn rm ./tika\*}} (remove everything but the KEYS file) 4. {{cp
    3. mkdir X.Y.Z
    4. rm -r X.Y.Z-1 (remove the former dist dev version folder)
    5. cp ../release/target/checkout/target/X.Y.Z/* X.Y.Z (copy the artifacts into the new dist.dev.tika/X.Y.Z directory)
    6. cd X.Y.Z
    7. mv CHANGES.txt \* .}} 5. {{mv CHANGES.txt CHANGES-X.Y.txt}} 6. {{cp KEYS .}} -- if it isn't there already copy the KEYS file to go along with the jars and other release artifacts; add your key to KEYS if you haven't already 7. {{svn add ./\*}} 8. {{svn commit -m "Add Tika X.Y RC#N artifacts."}} 9. See the file in {{Z.txt
    8. make sure the KEYS files is at dist.dev.tika/KEYS -- if it isn't there already copy the KEYS file to go along with the jars and other release artifacts; add your key to KEYS if you haven't already
    9. svn add X.Y.Z
    10. svn commit -m "Add Tika X.Y.Z RC#N artifacts."
13. See the file in target/checkout/target/vote.txt}}  - it contains the contents of the release email you should send (with subject {{\[VOTE\] Apache Tika X.Y .Z Release Candidate N}} to  to dev@tika.apache.org and to user@tika.apache.org). Ensure all URLs are correct and update the email to include links to the artifacts from #8 (including .sha, .md5 and sha512, .asc and CHANGES.txt)
14. If VOTE fails:
    1. Drop the new Tika staging repository on https://repository.apache.org
    2. Make an entry in CHANGES.txt which documents the subsequent changes between release candidates. This is extremely useful if the candidates are cut from master and master has moved on since the release candidate was originally cut.
    3. Go back to #1 10.
15. If the VOTE passes:
    Wiki Markup

    1. Send

    \

    1. [RESULT

    \

    1. ]

    \

    1. [VOTE

    \

    1. ]

    tallying

    1.  tallying the

    1. VOTE

    1. to

    1. dev@tika.apache.org

    1. and

    1. user@tika.apache.org

    2.

    2. Head
    1. over
    1. to
    1. Nexus
    1. from
    1. #6
    1. and
    1. "Release"
    1. the
    1. repository
    1. with
    1. message
    1. Apache
    1. Tika
    1. X.Y
    1. .Z release
    1. (and
    1. make
    1. sure
    1. the
    1. box
    1. is
    1. checked
    1. to
    1. auto
    1. drop).
    3. {{svn rm
    2. svn rm https://dist.apache.org/repos/dist/release/tika/
    \*}} 4. {{svn mv
    1. X.Y.Z-1* (delete the previous release)
    2. svn mv https://dist.apache.org/repos/dist/dev/tika/
    \* https
    1. X.Y.Z/* https://dist.apache.org/repos/dist/release/tika/
    }} 5. {{git clone
    1. X.Y.Z/*
    2. git clone https://github.com/apache/tika.git
    1. X.Y.Z-rcN
    }} 6. Add final tag for the release and delete the
    2. Add final tag for the release and delete the X.Y.Z-rcN
    1. tag:
       1. Look up commit for X.Y.Z-rcN commit: git
    show
    1. 1. rev-
    ref --abbrev=7 --tags
    1. 1. list \-n 1 1.25-rc2; let's say '
    d831efb
    1. 1. 0090ebac8e4ff4083a9c0c5d3dc55f545ad6f951'
    2.
    1. 2. git tag -a X.Y
    d831efb
    1. 1. .Z 0090eba -m "Tagging X.Y.Z release"
    3.
    1. 2. git tag -d X.Y.Z-rcN
    4.
    1. 2. git push --tags
    5.
    1. 2. git push origin :refs/tags/X.Y.Z-rcN
    
    Else:
16. Drop the new Tika staging repository on repository.apache.org. 2. Make an entry in CHANGES.txt which documents the subsequent changes between release candidates. This is extremely useful if the candidates are cut from master and master has moved on since the release candidate was originally cut. 3. Go back to #1.
12. Update the version of any unreleased modules (like tika-dotnet). 13.
17. Update Tika site.
    1. Update parent version number in the site pom.xml file.
    2. 2. Update the Documentation section of src/site/site.xml. Make sure to add the collapse tag for the now old version. 3.
    3. Update the "Full List of Supported Formats" section of the formats page of version X.Y.Z using TIKA-411; e.g. java -jar release-X.Y./masterZ/tika-app/target/tika-app-1.15.jar --list-parser-details-apt >> src/site/apt/1.15/formats.apt 4.
    4. Update the version towards the top of X.Y.Z/formats.apt: src/site/apt/1.15/formats.apt
    5. Update the version number in the dependency examples and the commandline options in the new X.Y.Z src/site/apt/X.Y.Z/gettingstarted.apt page 5.
    6. Update src/site/resources/doap.rdf with the new release. 6.
    7. Create a new example and formats page (truncated to where step 15.3 should go next time) for the next version (but don't list it in the sidebar).
    8. Update src/site/apt/ 7. Update index.apt.vm to include the announcement . 8. Update index.apt Generate the contributor list with and change link to previous release CHANGES.txt file (from https://dist.apache.org/repos/dist/release/tika/X.Y.Z/CHANGES-X.(Y-1).txt to https://githubarchive.apache.comorg/chrismattmanndist/tika/X.(Y-1)/CHANGES-X.(Y-1).txt).
    9. Update src/site/apt/X.Y.Z/index.apt.
       1. Generate the contributor list with apachestuff/blob/master/extract-tika-contribs 2. Generate the issues list by running https://github.com/chrismattmann/apachestuff/blob/master/extract-tika-issues.pycontribs (I use wget and then apply the pipe logic)
       2. against CHANGES.txt. 3. Update the link to the JIRA query (e.g. https://s.apache.org/XowY) via Apache's shortening service https://s.apache.org 9.
       3. Generate the issues list by running https://github.com/chrismattmann/apachestuff/blob/master/extract-tika-issues.py CHANGES-X.Y.Z.txt output.txt X.Y.Z  (I have local mods to allow this to work with python3. I should open a PR...)
          
    10. Generate the Javadoc.
        1. unzip tika-X.Y.Z-src.zip

        2. cd master 2. mvn javadoc:aggregate 3.
        3. mkdir .../tika-site/publish/X.Y.Z
        4. mv target/site/apidocs .../tika/-site/publish/X.Y.Z/api 10.
    11. Update tika-server documentation (We haven't had a license for a good while. This is no longer active):
        1. In the unzipped src release, cd tika-server
        2. mvn -Pserver install (in the tika-server module to build the miredot documentation 2. )
        3. mkdir .../tika-site/src/site/resources/X.Y.Z
        4. rm -r / tika/-site/src/site/resources/X.Y.Z/miredot (if it exists?!)
        5. cp -r 3. cp /tika/master/ tika-server/target/miredot .../tika/site/src/site/resources/X.Y
           11. .Z
    12. Update the downloads section (Unless you're supporting two major versions at a time, this should be automatic).
    13. Check everything looks good with mvn site:run. 12.
    14. mvn clean install
    15. 13. svn add <any files changed for site> (probably: src/site/apt/X.Y.Z and src/site/resources/X.Y.Z/miredot) && svn commit -m "Update website for X.Y.Z release." 14.
    16. On the Tika JIRA, "release" version X.Y.Z and update any (new) straggler X.Y.Z issues to X.(Y+1). 15.
18. Send announcements to user@tika.apache.org, dev@tika.apache.org, and announce@apache.org lists (from your Apache email) - see httphttps://s.apache.org/2hCytaod for a sample. If editing and resending an earlier email, but edited, make sure that you use text format and/or make sure that editing the links updates the underlying links and not just the anchor text. NOTE FOR THE 1.18/2.0 RELEASE and subsequent releases: Change these lines: Apache Tika is available in source form from the following download page: http://www.apache.org/dyn/closer.cgi/tika/apache-tika-1.6-src.zip to Apache Tika is available on the download page: http://tika.apache.org/download.html Change the link to the keys from https://people.apache.org/keys/group/tika.asc to https://www.apache.org/dist/tika/KEYS
19. After sending the announcement, update the shortened url above for this new sample!
16. Log on to https://reporter.apache.org/addrelease.html?tika and add the release data (version and date) to the database

NOTE: if anything goes wrong during mvn:release-prepare or mvn release:perform:

1. mvn release:release-clean
2. Undo any commits that changed the version number
3. Delete the X.Y.Z-rcN tag: git git push --delete origin X.Y.Z-rcN && git tag --delete X.Y.Z-rcN

After the release, consider making a PR to make the upgrade in Apache Solr. See UpgradingTikaInSolr.

After the release, announce any CVEs that were fixed and update Securitysecurity.apt and republish the site.

After the project release

Release the Docker Image

Release Process for tika-docker

Release the Helm Chart

Release Process for tika-helm