Page History

...

Setup release branches

Anchor
setup-release-branches setup-release-branches

 (Release Manager)

First, update the RELEASE_NOTES in incubator-streampipes (dev) by adding significant new features from JIRA that correspond to the new release version.

incubator-streampipes

Update active UI modules.

Depending on the features that should be released, the UI config for active modules needs to be updated. If the feature set changes, open the ui/deployment/rel/config.yml file and edit the list of active modules:

login:
  backgroundImage: 'deployment/rel/img/background.png'
  logo: 'deployment/rel/img/logo.png'
  logo-right: 'deployment/rel/img/sp-logo-right-white.png'
modules:
  - spEditor
  - spPipelines
  - spConnect
  - spDashboard
  - spAppOverview
  - spAdd
  - spMyElements
  - spDataExplorer
  - spConfiguration

Commit and push these changes to dev

If needed, change the current SNAPSHOT version to the version that is planned for the upcoming release:

# Make sure to only use a SNAPSHOT version here
# THIS STEP IS ONLY NEEDED IF THE NEXT RELEASE VERSION DIFFERS FROM THE CURRENT SNAPSHOT VERSION (e.g., when jumping from 0.90.0 to 0.95.0)

# Make sure to only use a SNAPSHOT version here

mvn versions:set -DnewVersion=0.66.0-SNAPSHOT

# Check the output and briefly check if all changed version numbers in the POM files are ok

mvn versions:commit

# Manually change the project version in the package.json file in the ui/ folder.

# Push the changes to dev

...

The maven-release-plugin will ask you to provide the version number of the next development version (use the new minor version, not the suggested patch version). Afterwards, a new branch will be created and the version number in dev is increased to the next development version. All changes will be automatically pushed by the plugin.

incubator-streampipes-extensions

The next step is to prepare the extensions project for a release. The required steps are almost identical to the previous steps:

API Docs

Update the streampipes version within the API docsIf needed, change the current SNAPSHOT version to the version that is planned for the upcoming release:

# Go into incubator-streampipes-extensions
# Make sure to only use a SNAPSHOT version here

mvn versions:set -DnewVersion=0.66.0-SNAPSHOT

# Check the output and briefly check if all changed version numbers in the POM files are ok

mvn versions:commit

# Open the incubator-streampipes-extensions parent pom and change the parent version and the streampipes.version property to the SNAPSHOT version.

# Commit and push the changes to dev

Afterwards, create a new branch for the upcoming release candidate:

...

...

/streampipes-service-core/src/main/resources/openapi.yaml in dev

# adapt the streampipes version 
openAPI:
  info:
    version: '0.91.0'

Checkout dev and change the aforementioned versions to the next release version (e.g., 0.67.0-SNAPSHOT)

Prepare test setup

Anchor
prepare-test-setup prepare-test-setup

 (Release Manager)

...

In case modules are changed, update the docker-compose files in incubator-streampipes and incubator-streampipes-extensions streampipes project by adding/modifying/removing defined services.

...

The following build tools need to be installed to properly test a release candidate:

• JDK (tested with JDK 8+starting from v0.70, JDK 17 is the new minimum java version)
• Maven (tested with 3.6)
• NodeJS + NPM (tested with v12+/ v6+)
• Docker + Docker-Compose

Checkout release branches

Anchor
checkout-release-branches checkout-release-branches

 (Community)

First, create a new folder and checkout the release branches of both projects incubator- streampipes and incubator-streampipes-extensions:

# Clone & checkout a clean copy from the release branch

mkdir test-streampipes-rc && cd test-streampipes-rc
git clone git@github.com:apache/incubator-streampipes.git
git clone git@github.com:apache/incubator-streampipes-extensions.git

# You can also clone the projects using HTTPSALTERNATIVE: git clone https://github.com/apache/incubator-streampipes.git

cd incubator-streampipes
# Change VERSION to the current release branch
git checkout rel/VERSION

cd ../incubator-streampipes-extensions
# Change VERSION to the current release branch
git checkout rel/VERSION

Setup test environment

Anchor
setup-test-environment setup-test-environment

 (Community)

Now we'll build the system from source and run a test instance in Docker. Note that the docker-compose file used here should only be used for final release testing, as no volumes are defined for persistence.

incubator-streampipes

Build backend & UI

From the parent folder of the core, do the following to build the Java-based core:

# Openin incubator-streampipesdownload folderdirectory
cd incubator-streampipes

# Run Maven Build, use install to make core artifacts available to the extensions build later
mvn clean install

mvn clean package

This build should be successful.

The next step is to build the UI, switch to the ui directory and do the following:

# Run NPM Build
cd ui

npm install
# If this results in an error, do npm run build-rc

# Go back to the main directory of incubator-streampipes

docker-compose up

# This will build the Docker images for the streampipes-backend and the streampipes-connect-master modules. In addition, it will start several 3rd party Docker containers that are 
# mandatory for StreamPipes, e.g., CouchDB and Consul. Again, be aware that this docker-compose configuration does not include volume persistency and should therefore only be used for 
# short-term testing.

incubator-streampipes-extensions

Now we'll build and start the extensions project:

# Open incubator-streampipes-extensions folder
cd incubator-streampipes-extensions

# Run Maven Build
mvn clean package
docker-compose up

install --legacy-peer-deps

npm run build

Start test system 

If you want to test the system, there is a convenience docker-compose file that will help you starting the release candidate for testing.

Go back to the main directory of the core release artifact (where the docker-compose.yml file is located) and do the following:

# this might be optional - seems to only occur in rare cases on Windows systems

# Set the correct EOL encoding for the UI entrypoint file (docker-entrypoint.sh) file to LF:
awk 'BEGIN{RS="^$";ORS="";getline;gsub("\r","");print>ARGV[1]}' ui/docker-entrypoint.sh

# Alternative: Open the file ui/docker-entrypoint.sh, change the eol encoding to "LF" (e.g., in Notepad++ by clicking on "Windows CRLF" in the bottom status bar).

docker-compose up --build -d

# Use docker-compose up to run it in foreground
# Docker images will be locally built

Once you're done with that, proceed to the next step. This will start all extensions (except Flink modules) and you're ready for testing!

Test system

Anchor
test-system test-system

 (Community)

...