...

Pre-release process

General Tasks

Agree on upcoming release

Anchor
agree-on-upcoming-release agree-on-upcoming-release

 (Community)

At some point in time, some member of the community thinks that it is time for a new release. If you think so, write an email to the dev@streampipes.apache.org list and ask if there is consensus on performing the next release.

Appoint release manager

Anchor
appoint-release-manager appoint-release-manager

 (Community)

Every release must have a release manager. Following up the agreement, some member of the Apache StreamPipes PPMC needs to volunteer as a release manager (RM). Usually, this is done by replying to the previous email on the upcoming release.

Agree on feature set for next release

Anchor
agree-on-feature-set agree-on-feature-set

 (Community)

Once a release manager has been found, the community discusses features that are not yet finished but should be part of the next release.

Prepare release testing

Anchor
prepare-release-testing prepare-release-testing

Create test checklist

Anchor
create-test-checklist create-test-checklist

 (Release Manager)

To track things that have been tested by community members, a checklist can be used to indicate things that work or do not work.

Create a new checklist by copying the template from Pre-Release Checklist to a new page that indicates the next release version.

Setup release branches

Anchor
setup-release-branches setup-release-branches

First, update the RELEASE_NOTES in incubator-streampipes (dev) by adding significant new features from JIRA that correspond to the new release version.

incubator-streampipes

Update active UI modules.

...

 (Release Manager)

login:
  backgroundImage: 'deployment/dev/img/background.png'
  logo: 'deployment/dev/img/logo.png'
  logo-right: 'deployment/dev/img/sp-logo-right-white.png'
modules:
  - spEditor
  - spPipelines
  - spConnect
  - spDashboard
  - spAppOverview
  - spAdd
  - spMyElements
  - spDataExplorer
  - spConfiguration

If needed, change the current SNAPSHOT version to the version that is planned for the upcoming release:

# Make sure to only use a SNAPSHOT version here
# THIS STEP IS ONLY NEEDED IF THE NEXT RELEASE VERSION DIFFERS FROM THE CURRENT SNAPSHOT VERSION (e.g., when jumping from 0.90.0 to 0.95.0)

# Make sure to only use a SNAPSHOT version here

mvn versions:set -DnewVersion=0.66.0-SNAPSHOT

# Check the output and briefly check if all changed version numbers in the POM files are ok

mvn versions:commit

# Manually change the project version in the package.json file in the ui/ folder.

# Push the changes to dev

...

The maven-release-plugin will ask you to provide the version number of the next development version (use the new minor version, not the suggested patch version). Afterwards, a new branch will be created and the version number in dev is increased to the next development version. All changes will be automatically pushed by the plugin.

API Docs

Update the streampipes version within the API docs:

# Go to /streampipes-service-core/src/main/resources/openapi.yaml in dev

# adapt the streampipes version 
openAPI:
  info:
    version: '0.91.0'

Checkout dev and change the aforementioned versions to the next release version (e.g., 0.67.0-SNAPSHOT)

Prepare test setup

Anchor
prepare-test-setup prepare-test-setup

 (Release Manager)

During the test phase, StreamPipes contributors should test the pre-release candidate (the branch we've just built) based on the provided docker-compose files for validation.

In case modules are changed, update the docker-compose files in streampipes project by adding/modifying/removing defined services.

Perform technical release readiness assessment

Anchor
perform-technical-release-readiness-assessment perform-technical-release-readiness-assessment

Requirements

The following build tools need to be installed to properly test a release candidate:

• JDK (starting from v0.70, JDK 17 is the new minimum java version)
• Maven (tested with 3.6)
• NodeJS + NPM (tested with v12+/ v6+)
• Docker + Docker-Compose

Checkout release branches

Anchor
checkout-release-branches checkout-release-branches

 (Community)

First, create a new folder and checkout the release branches of streampipes:

# Clone & checkout a clean copy from the release branch

mkdir test-streampipes-rc && cd test-streampipes-rc
git clone git@github.com:apache/streampipes.git

# ALTERNATIVE: git clone https://github.com/apache/streampipes.git 

cd streampipes
# Change VERSION to the current release branch
git checkout rel/VERSION

Setup test environment

Anchor
setup-test-environment setup-test-environment

 (Community)

Now we'll build the system from source and run a test instance in Docker. Note that the docker-compose file used here should only be used for final release testing, as no volumes are defined for persistence.

Build backend & UI

From the parent folder of the core, do the following to build the Java-based core:

# in download directory
mvn clean package

This build should be successful.

The next step is to build the UI, switch to the ui directory and do the following:

# cd ui

npm install
# If this results in an error, do npm install --legacy-peer-deps

npm run build

Start test system 

If you want to test the system, there is a convenience docker-compose file that will help you starting the release candidate for testing.

Go back to the main directory of the core release artifact (where the docker-compose.yml file is located) and do the following:

# this might be optional - seems to only occur in rare cases on Windows systems

# Set the correct EOL encoding for the UI entrypoint file (docker-entrypoint.sh) file to LF:
awk 'BEGIN{RS="^$";ORS="";getline;gsub("\r","");print>ARGV[1]}' ui/docker-entrypoint.sh

# Alternative: Open the file ui/docker-entrypoint.sh, change the eol encoding to "LF" (e.g., in Notepad++ by clicking on "Windows CRLF" in the bottom status bar).

docker-compose up --build -d

# Use docker-compose up to run it in foreground
# Docker images will be locally built

Once you're done with that, proceed to the next step. 

Test system

Anchor
test-system test-system

 (Community)

Go to your browser and open the StreamPipes UI at localhost:80.

Perform the installation steps (all pipeline elements should be automatically installed) and test the system according to the Pre-Release Checklist. Mark everything you have checked in the pre-release checklist that has been created for the upcoming release.

Fix critical issue

Anchor
fix-critical-issue fix-critical-issue

 (Community)

If you find a release-blocking issue during testing, do the following steps:

• Create an issue in JIRA
• If you plan to fix the issue by yourself:
  • Fix the issue in the release branch (e.g., rel/0.66.0)

Cherry-pick branches into dev (Community/Committer)

• All changes made to the release branch should be cherry-picked back into dev (git log / git cherry-pick COMMIT_HASH)

Perform legal release readiness assessment

Checkout release branches (Community)

Perform legal checks (Community)

Fix legal issue (Community)

Cherry-pick branches into dev (Community/Committer)