Page History

...

steps:
      - name: Temp check out parent_pom code
        uses: actions/checkout@v3
        with:
          ref: "my_parent_pom_branch"

      - name: Temp install parent_pom
        run: mvn clean install

CI utilities

we We have a collection of ci utilities that connectors should use.

https://github.com/apache/flink-connector-shared-utils/tree/ci_utilsThe ci

The CI utilities requires maintainers to think about against which Flink versions the connector should be tested. Most likely this is something like this:

• CI for PRs is tested against released versions of Flink, like 1.17.2 or 1.18.0. This enables if the change in the code would work against existing versions of Flink.
• CI for nightly/weekly builds are usually a combination of released versions of Flink, and against Flink snapshots.
  • For branches of the connector that have been released (like v1.0/v3.0), it makes sense to test against Flink SNAPSHOT versions. That allows you to check in the nightly/weekly builds if your connector is still working as expected against unreleased versions of Flink. 
  • The main  branch of the connector has most likely not been released yet. For those it makes sense to test against supported SNAPSHOT versions. With PRs, you should have already tested against released versions. With the nightly/weekly builds, it would allow you to test if your connector still works against unreleased versions of Flink. 

1. CI for PRs
   
   The push_pr.yml workflow can be used like this:
   

   Code Block
   jobs: jobs: compile_and_test: strategy: matrix: flink: [ 1.17.2 ] jdk: [ '8, 11' ] include: - flink: 1.18.1 jdk: '8, 11, 17' uses:

...

1. apache/flink-connector-shared-utils/.github/workflows/ci

...

1. .yml@ci_utils with: flink_version: ${{ matrix.flink }} jdk_version: ${{ matrix.jdk }} python_test: strategy: matrix: flink: [ 1.17.2, 1.18.1 ] uses: apache/flink-connector-shared-utils/.github/workflows/python_ci.yml@ci_utils with: flink_version: ${{ matrix.flink }}

   
   

2. CI for nightly/weekly checks
   
   The weekly.yml  can be used like this:
   
   

name: Nightly
on:
  schedule:
    - cron: "0 0 * * 0"
  workflow_dispatch:
jobs:
  compile_and_test:
    if: github.repository_owner == 'apache'
    strategy:
      matrix:
  1.16.0
      flink_url: https://dist.apache.org/repos/dist/release/flink/flink-1.16.0/flink-1.16.0-bin-scala_2.12.tgz
	  branch: v3.0branches: [{
          flink: 1.17-SNAPSHOT,
          branch: main
        }, {
          flink: 1.18-SNAPSHOT,
          jdk: '8, 11, 17',
          branch: main
        }, {
          flink: 1.19-SNAPSHOT,
          jdk: '8, 11, 17, 21',
          branch: main
        }, {
          flink: 1.17.1,
          branch: v3.0
        }, {
          flink: 1.18.0,
          branch: v3.0
        }]
    uses: apache/flink-connector-shared-utils/.github/workflows/ci.yml@ci_utils
    with:
      flink_version: ${{ matrix.flink_branches.flink }}
      connector_branch: ${{ matrix.flink_branches.branch }}
      cache_flink_binary: truejdk_version: ${{ matrix.flink_branches.jdk || '8, 11' }}
      run_dependency_convergence: false

Release utilities

We have a collection of release scripts that connectors should use.

...