...

An Apache StreamPipes release currently consists of three different one source releases:

• incubator-streampipes is the core of Apache StreamPipes
  incubator-streampipes-extensions contains several
  • This also contains all the extensions (> 20 adapters and >
  70
  • 100 pipeline elements)
  incubator-streampipes-installer contains installation
  • & the installer (containing files for
  Docker
  • docker, a CLI, and helm charts for
  running StreamPipes in
  • Kubernetes
  .
  • )

  Using the validation script

  ...

  The artifacts are available at https://dist.apache.org/repos/dist/dev/incubator/streampipes 

  Typically, this includes three one zip files (core, extensions and installer) along with checksum, signature, README, RELEASE_VALIDATION and RELEASE_NOTES.

  You can either download the release artifacts one by one, or use the convenient single line command below:

  TODO: Update download link

  Code Block
  language bash title Download release artifacts
  # download all release artifacts (core, extensions, installer) wget --recursive --no-host-directories -e robots=off --cut-dirs=5 --no-parent --reject "index.html*" https://dist.apache.org/repos/dist/dev/incubator/streampipes/

  ...

  Code Block
  title Build Maven
  # cd download-dir/apache-streampipes-#.##.#-incubating/ mvn clean install # we use install here as these dependencies will be later required by streampipes-extensions (with the core modules not yet being available on Maven Central at release validation time)package

  This build should be successful.

  ...

  Go back to the main directory of the core release artifact (where the docker-compose.yml file is located) and do the following:

  TODO: Check if location is still the same

  Code Block
  title Set proper line encoding in docker entrypoint
  # Set the correct EOL encoding for the UI entrypoint file (docker-entrypoint.sh) file to LF: awk 'BEGIN{RS="^$";ORS="";getline;gsub("\r","");print>ARGV[1]}' ui/docker-entrypoint.sh # Alternative: Open the file ui/docker-entrypoint.sh, change the eol encoding to "LF" (e.g., in Notepad++ by clicking on "Windows CRLF" in the bottom status bar).

  ...

  Once you're done with that, proceed to the next step. Before we test the whole system, we'll build and run the extensions.

  Step 3: Validate incubator-streampipes-extensions

  Open the extensions folder (extensions/{version}/{rc}.

  Repeat the following steps for the extensions project, similar as explained above:

  Verify signature

  Code Block
  language bash title Verify signature
  gpg --verify apache-streampipes-extensions-{current-full-version}-incubating-source-release.zip.asc apache-streampipes-extensions-{current-full-version}-incubating-source-release.zip # example gpg --verify apache-streampipes-extensions-0.68.0-incubating-source-release.zip.asc apache-streampipes-extensions-0.68.0-incubating-source-release.zip

  Verify checksum

  Code Block
  title Verify checksum
  sha512sum -c apache-streampipes-extensions-{current-full-version}-incubating-source-release.zip.sha512 # example: sha512sum -c apache-streampipes-extensions-0.68.0-incubating-source-release.zip.sha512

  Make sure the output says OK.

  Unzip the source file

  Unzip and switch to the parent directory (where the README and pom files are located)

  Code Block
  title Unzip
  unzip apache-streampipes-extensions-{current-full-version}-incubating-source-release.zip cd apache-streampipes-extensions-{current-full-version}-incubating/ # example: unzip apache-streampipes-extensions-0.68.0-incubating-source-release.zip cd apache-streampipes-extensions-0.68.0-incubating/

  Verify legal information and required files

  •  Verify the existence of LICENSE, NOTICE, README, RELEASE_NOTES and RELEASE_VALIDATION files in the extracted source bundle.
  •  Check the LICENSE file
  •  Check the NOTICE file (e.g., make sure the year is correct)

  ...

  Run RAT

  Code Block
  title Run RAT
  mvn rat:rat

  Search for SNAPSHOT dependencies

  Search for any unexpected SNAPSHOT dependencies in the bundle, e.g.:

  Code Block
  title Search for SNAPSHOT dependencies
  grep -rnw './' -e 'SNAPSHOT' # Check the output

  Build extensions

  Build the extensions project as follows:

  Code Block
  title Build with Maven
  mvn clean package

  Start test system

  Go back to the main directory of the extensions release artifact (where the docker-compose.yml file is located) and do the following:

  Code Block
  title Build Docker images and start
  docker-compose up --build -d # Use docker-compose up to run it in foreground # Docker images will be locally built

  See the instructions below under "Step 4: Test" to test the release candidate.

  Step 3: incubator-streampipes-installer

  Open the installer folder (installer/{version}/{rc}).

  The installer contains no files that need to be built, just some bash scripts, docker-compose files and helm charts.

  Repeat the following steps for the installer project, similar as explained above:

  Verify signature

  Code Block
  language bash title Verify signature
  gpg --verify apache-streampipes-installer-{current-full-version}-incubating-source-release.zip.asc apache-streampipes-installer-{current-full-version}-incubating-source-release.zip # example gpg --verify apache-streampipes-installer-0.68.0-incubating-source-release.zip.asc apache-streampipes-installer-0.68.0-incubating-source-release.zip

  Verify checksum

  Code Block
  title Verify checksum
  sha512sum -c apache-streampipes-installer-{current-full-version}-incubating-source-release.zip.sha512 # example: sha512sum -c apache-streampipes-installer-0.68.0-incubating-source-release.zip.sha512

  Make sure the output says OK.

  Unzip the source file

  Unzip and switch to the parent directory.

  Code Block
  title Unzip
  unzip apache-streampipes-installer-{current-full-version}-incubating-source-release.zip cd apache-streampipes-installer-{current-full-version}-incubating/ # example: unzip apache-streampipes-installer-0.68.0-incubating-source-release.zip cd incubator-streampipes-installer/

  Verify legal information and required files

   

  ...

  Step 2

  ...

  : Test

  If you have started the core and extensions in Docker as explained in the previous step, you can now switch to the user interface to do functional testing:

  Open your browser and go to http://localhost (port 80) to get to the StreamPipes UI. You'll see the setup screen where you can start the setup procedure using a test user, e.g.,login screen:

  • Default dummy user email:  streampipes@apacheadmin@streampipes.apache.org
  • dummy Default password: passwordadmin

  Once you are logged in, you can test the functionalities of the softwareIf all goes well, the StreamPipes core as well as extensions are initialized and you should only see "checkmarks". After this step, click "Go to Login Page" and login to the system with the before specified credentials.

  Info
  title Testing with the release validation docker-compose file
  The docker-compose.yml file that is part of the source is only intended for developers. There is no persistence enabled, when you stop the Docker containers, all information will be lost. Users are intended to use the installer.

  ...

  •  Download all staged artifacts under the url specified in the release vote email
  •  Verify the signature is correct
  •  Check if the signature references an Apache email address
  •  Verify the SHA512 checksum
  •  Unzip the archive
  •  Verify the existence of LICENSE, NOTICE, README, RELEASE_NOTES, RELEASE_VALIDATION files in the extracted source bundle.
  •  Run RAT
  •  Search for SNAPSHOT dependencies
  •  Build Backend
  •  Build UI
  •  Build Extensions
  •  Build and Run Test system on Docker

  ...