Adding a Python version

  1. Add a Beam Python container for the new python version.
    1. https://github.com/apache/beam/tree/master/sdks/python/container
  2. Add a new Python version to different test-suites.
    1. Tox test suites - link
    2. All the required Gradle tasks such as pre-commits, post-commits etc.
    3. Make sure all the runners support the new Python version.
  3. Add the Github actions suite for the new Python version.
    1. https://github.com/apache/beam/blob/master/.github/workflows/python_tests.yml
  4. Add the support for building wheels for the new Python version.
    1. https://github.com/apache/beam/blob/master/.github/workflows/build_wheels.yml
  5. Update the upper limit in __init__.py with the next major Python version.
  6. Update any code that needs changes for the new Python version.
  • If there is a new feature update or there is a regression when adding a new Python version, please file an issue.
    • All the unit tests and Integration tests must pass before merging the new version. 
    • If you are a non-committer, please ask the committers to run a seed job on your PR to test all the new changes.

Pull request for adding python 3.9 to Apache Beam: https://github.com/apache/beam/pull/16008

Removing a Python version

  1. Bump the python version in setup.py and update the python version warning in __init__.py. 
  2. Delete the unsupported Python version containers from sdks/python/container
  3. Remove the test-suites for the unsupported python version from
    1. Remove unsupported py{version} from the following files/directories
      1. sdks/python/test-suites/gradle.properties
      2. apache_beam/testing/tox
      3. apache_beam/testing/dataflow
      4. apache_beam/testing/direct
      5. apache_beam/testing/portable
    2. Remove the unsupported python version gradle tasks from
      1. beam/.test-infra/{DSL_SCRIPT}.groovy
      2. build.gradle.kts
      3. settings.gradle.kts
      4. buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
    3. Remove the support for building wheels and source distributions for the unsupported Python version from .github/workflows/build_wheels.yml
    4. Remove the unsupported Python version from GitHub actions unit tests at .github/workflows/python_tests.yml
    5. Remove the unsupported Python version from sdks/python/tox.ini
  4. Clean up any code that applies to unsupported Python version[Optional]


Pull request for removing Python 3.6 support from Apache Beam: https://github.com/apache/beam/pull/17252

  • No labels