Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Use the following code:

    Code Block
    languagebash
    # Initialize virtual environment called "env" in ~/.virtualenvs or any other directory. (Consider using pyenv, to manage the python version as well as installed packages in your virtual environment)
    $ python3 -m venv ~/.virtualenvs/env
    
    # Activate virtual environment.
    $ . ~/.virtualenvs/env/bin/activate
    
    # Upgrade other tools. (Optional)
    pip install --upgrade pip
    pip install --upgrade setuptools
    
    # Install setup.py requirements.
    (env) $ pip install -r build-requirements.txt
    
    # Install Apache Beam package in editable mode.
    (env) $ pip install -e .[gcp,test]
    
    

    For certain systems, particularly Macs with M1 chips, this installation method may not generate urns correctly. If running python gen_protos.py  doesn't resolve the issue, consult https://github.com/apache/beam/issues/22742#issuecomment-1218216468 for further guidance.

...

Code Block
languagebash
cd sdks/python/
pip install build && python setup.py  -m build --sdist

We will use the tarball built by this command in the --sdk_location parameter.

...

  1. Click on a recent `Build python source distribution and wheels job` that ran successfully on the github.com/apache/beam master branch from this list
  2. Click on List files on Google Cloud Storage Bucket on the right-side panel.
  3. Expand List file on Google Cloud Storage Bucket in the main panel.
  4. Locate and Download the ZIP file. For example, apache-beam-2.4852.0.dev0.zip from tar.gz from GCS.
  5. Install the downloaded zip file. For example:

    Code Block
    languagebash
    titleSimpleTest
    pip install apache-beam-2.4852.0.dev0.ziptar.gz
    # Or, if you need extra dependencies:
    pip install apache-beam-2.4852.0.dev0.tar.zipgz[aws,gcp]


  6. When you run your Beam pipeline, pass in the --sdk_location flag pointed at the same ZIP file. 


    Code Block
    languagebash
    titleSimpleTest
    --sdk_location=apache-beam-2.2552.0.dev0.ziptar.gz


How to update dependencies that are installed in Python container images 

...