...
| Code Block |
|---|
| language | bash |
|---|
| title | confirm docker is installed |
|---|
| linenumbers | true |
|---|
|
$ docker -v
Docker version 18.09.3, build 774a1f4 |
...
Run a test against containers
Running precommit tests at local
| Code Block |
|---|
| language | bash |
|---|
| title | Python |
|---|
| linenumbers | true |
|---|
|
$ ./gradlew :sdks:python:test-suites:portable:py2:preCommitPy2
$ ./gradlew :sdks:python:test-suites:portable:py35:preCommitPy35
$ ./gradlew :sdks:python:test-suites:portable:py35:preCommitPy36
$ ./gradlew :sdks:python:test-suites:portable:py35:preCommitPy37 |
| Code Block |
|---|
| language | bash |
|---|
| title | Java |
|---|
| linenumbers | true |
|---|
|
$ ./gradlew :javaPreCommitPortabilityApi --continue --info |
| Code Block |
|---|
| language | bash |
|---|
| title | Go |
|---|
| linenumbers | true |
|---|
|
$ ./gradlew :goPreCommit |
Run postcommit tests at local
| Code Block |
|---|
| language | bash |
|---|
| title | Python |
|---|
| linenumbers | true |
|---|
|
$ ./gradlew :python2PostCommit
$ ./gradlew :python35PostCommit
$ ./gradlew :python36PostCommit
$ ./gradlew :python37PostCommit |
| Code Block |
|---|
| language | bash |
|---|
| title | Java |
|---|
| linenumbers | true |
|---|
|
$ ./gradlew :javaPostCommitPortabilityApi --continue --info |
| Code Block |
|---|
| language | bash |
|---|
| title | Go |
|---|
| linenumbers | true |
|---|
|
$ ./gradlew :goPostCommit |
Publish
Publishing images to gcr.io/beam requires permissions in apache-beam-testing project.
Please note this will create new images (not used for above testings). Since they are created from the same code, we assume exactly same and if all tests passed with the new created images then assume images created above also pass the tests.
Set repository and tag as variables:
...
| Code Block |
|---|
| language | bash |
|---|
| title | For release |
|---|
| linenumbers | true |
|---|
|
export REPOSITORY=gcr.io/apache-beam-testing/beam/sdk/release
export TAG=2.15.0 |
To build and push docker images run:
| Code Block |
|---|
| language | bash |
|---|
| title | Python |
|---|
| linenumbers | true |
|---|
|
$ pwd
[...]/beam/
$ ./gradlew :sdks:python:container:py2:docker -Pdocker-repository-root=$REPOSITORY -Pdocker-tag=$TAG --info
$ ./gradlew :sdks:python:container:py35:docker -Pdocker-repository-root=$REPOSITORY -Pdocker-tag=$TAG --info
$ ./gradlew :sdks:python:container:py36:docker -Pdocker-repository-root=$REPOSITORY -Pdocker-tag=$TAG --info
$ ./gradlew :sdks:python:container:py37:docker -Pdocker-repository-root=$REPOSITORY -Pdocker-tag=$TAG --info
|
...
| Code Block |
|---|
| language | bash |
|---|
| title | Java |
|---|
| linenumbers | true |
|---|
|
$ ./gradlew :sdks:java:container:dockerdockerPush -Pdocker-repository-root=$REPOSITORY -Pdocker-tag=$TAG --info |
...
| Code Block |
|---|
| language | bash |
|---|
| title | Go |
|---|
| linenumbers | true |
|---|
|
$ ./gradlew :sdks:go:container:dockerdockerPush -Pdocker-repository-root=$REPOSITORY -Pdocker-tag=$TAG --info |
This produces local images named
| Code Block |
|---|
| language | bash |
|---|
| title | docker images list |
|---|
| linenumbers | true |
|---|
|
$ docker images
$REPOSITORY/python2.7:$TAG
$REPOSITORY/python3.5:$TAG
$REPOSITORY/python3.6:$TAG
$REPOSITORY/python3.7:$TAG
$REPOSITORY/java:$TAG
$REPOSITORY/go:$TAG |
They can be examined by running docker commands:
| Code Block |
|---|
| language | bash |
|---|
| linenumbers | true |
|---|
|
$ docker image ls
$ docker run --entrypoint bash -it $REPOSITORY/python3.5:$TAG |
Run a test against locally built container
Please note all following tests create new images each time, so they don't use the images we created above, however, since they are created from the same code, so we assume images are exactly same and if all tests passed with the new created images then assume images created above also pass the tests.
Running precommit tests at local
| Code Block |
|---|
| language | bash |
|---|
| title | Python |
|---|
| linenumbers | true |
|---|
|
$ ./gradlew :sdks:python:test-suites:portable:py2:preCommitPy2
$ ./gradlew :sdks:python:test-suites:portable:py35:preCommitPy35
$ ./gradlew :sdks:python:test-suites:portable:py35:preCommitPy36
$ ./gradlew :sdks:python:test-suites:portable:py35:preCommitPy37 |
| Code Block |
|---|
| language | bash |
|---|
| title | Java |
|---|
| linenumbers | true |
|---|
|
$ ./gradlew :javaPreCommitPortabilityApi --continue --info |
| Code Block |
|---|
| language | bash |
|---|
| title | Go |
|---|
| linenumbers | true |
|---|
|
$ ./gradlew :goPreCommit |
Run postcommit tests at local
| Code Block |
|---|
| language | bash |
|---|
| title | Python |
|---|
| linenumbers | true |
|---|
|
$ ./gradlew :python2PostCommit
$ ./gradlew :python35PostCommit
$ ./gradlew :python36PostCommit
$ ./gradlew :python37PostCommit |
| Code Block |
|---|
| language | bash |
|---|
| title | Java |
|---|
| linenumbers | true |
|---|
|
$ ./gradlew :javaPostCommitPortabilityApi --continue --info |
| Code Block |
|---|
| language | bash |
|---|
| title | Go |
|---|
| linenumbers | true |
|---|
|
$ ./gradlew :goPostCommit |
Publish
Publishing images to gcr.io/beam requires permissions in apache-beam-testing project.
| Code Block |
|---|
| language | bash |
|---|
| title | Python |
|---|
| linenumbers | true |
|---|
|
$ docker push $REPOSITORY/python2.7:$TAG
$ docker push $REPOSITORY/python3.5:$TAG
$ docker push $REPOSITORY/python3.6:$TAG
$ docker push $REPOSITORY/python3.7:$TAG |
| Code Block |
|---|
| language | bash |
|---|
| title | Java |
|---|
| linenumbers | true |
|---|
|
$ docker push $REPOSITORY/java:$TAG |
| Code Block |
|---|
| language | bash |
|---|
| title | Go |
|---|
| linenumbers | true |
|---|
|
$ docker push $REPOSITORY/go:$TAG |
Release Images Validation
...
