Page History

...

• You can download, build and install CPython from sources.
• If you are an Ubuntu user, you could add a third-party repository 'Deadsnakes' and install the missing versions via apt. If you install from Deadsnakes, make sure to also install python#.#-dev, python#.#-venv and python#.#-distutils packages.
• You can use PyEnv to download and install Python versions (Recommended).
  Installation steps may look  as follows:
  1. Follow the steps below in How to setup pyenv.

  2. Install Python intepreter for each supported Python minor version. For example:

     Code Block
     language bash
     pyenv install 3.7.10 pyenv install 3.8.9 pyenv install 3.9.4 pyenv install 3.10.7 pyenv install 3.11.3

     For major.minor.patch versions currently used by Jenkins cluster, see  Current Installations.

  3. Make installed interpreters available in your shell by running

     Code Block
     language bash
     pyenv global 3.8.9 3.7.10 3.9.4 3.10.7 3.11.3

     
     

  4. (OPTIONAL) Pyenv will sometimes fail to make these interpreters directly available without a local configuration. If you see errors trying to use python3.x , then run also pyenv local  

     Code Block
     language bash
     pyenv local 3.8.9 3.7.10 3.9.4 3.10.7 3.11.3

     
     

After these steps, all python3.x  interpreters should be available in your shell. The first version in the list passed to pyenv global will be used as default python / python3 interpreter if the minor version is not specified.

...

(env) $ pip install tox
(env) $ tox -c tox.ini tox run -e py38-cloud  # all tests
(env) $ tox -c tox.ini run -e py38 -- -k test_progress

...

1. Click on a recent `Build python source distribution and wheels job` that ran successfully on the github.com/apache/beam master branch from this list. 
2. Click on List files on Google Cloud Storage Bucket on the right-side panel.
3. Expand List file on Google Cloud Storage Bucket in the main panel.
4. Locate and Download the ZIP file. For example, apache-beam-2.2548.0.dev0.zip from GCS.
• It’s simplest to download the file using your browser by replacing the prefix “gs://” with “https://storage.googleapis.com/” . For example,  https://storage.googleapis.com/beam-wheels-staging/master/02bf081d0e86f16395af415cebee2812620aff4b-207975627/apache-beam-2.25.0.dev0.zip
• Or follow these instructions to download using the gsutil command-line tool. 

5. Install the downloaded zip file. For example:
   

   Code Block
   language bash title SimpleTest
   pip install apache-beam-2.2548.0.dev0.zip # Or, if you need extra dependencies: pip install apache-beam-2.2548.0.dev0.zip[aws,gcp]

   
   

6. When you run your Beam pipeline, pass in the --sdk_location flag pointed at the same ZIP file. 

   
   

   Code Block
   language bash title SimpleTest
   --sdk_location=apache-beam-2.25.0.dev0.zip

   
   

...

NOTE for RELEASE MANAGERS: The updated Python dependency files must be merged into Beam's master  branch before cutting the release branch.

Errors 

You may see the following error with a particular python version like Python 3.6. 

> Task :sdks:python:container:py36:generatePythonRequirements FAILED
+ ENV_PATH=/usr/local/google/home/yeandy/beam/build/python36_requirements_gen
+ rm -rf /usr/local/google/home/yeandy/beam/build/python36_requirements_gen
+ python3.6 -m venv /usr/local/google/home/yeandy/beam/build/python36_requirements_gen
Error: Command '['/usr/local/google/home/yeandy/beam/build/python36_requirements_gen/bin/python3.6', '-Im', 'ensurepip', '--upgrade', '--default-pip']' died with <Signals.SIGSEGV: 11>.

FAILURE: Build failed with an exception.

* Where:
Script '/usr/local/google/home/yeandy/beam/sdks/python/container/common.gradle' line: 37

* What went wrong:
Execution failed for task ':sdks:python:container:py36:generatePythonRequirements'.
> Process 'command 'sh'' finished with non-zero exit value 1

You may see that the pip command will lead to segmentation fault as well. If this happens, remove the python version from pyenv, and reinstall the version like this.that the pip command will lead to segmentation fault as well. If this happens, remove the python version from pyenv, and reinstall the version like this.

CFLAGS="-O2" pyenv install 3.67.12

There have been issues with older Python versions. See here for details.