Java Tips

How do I automatically format code and avoid spotless errors?

You can set up a git precommit hook to always autoformat code, by putting this in .git/hooks/pre-commit and setting the executable bit.

    #!/bin/sh
    set -e
    ./gradlew spotlessApply

If you haven't used git hooks, the docs are here: https://git-scm.com/docs/githooks.

Using --no-verify will skip it and `chmod u-x` will disable it.

How do I run a single test?

./gradlew :beam-examples-java:test --tests org.apache.beam.examples.subprocess.ExampleEchoPipelineTest --info

Running Java Dataflow Hello World pipeline with compiled Dataflow Java worker.

You can actually dump multiple definitions for gcp project name and temp folder. They are present, since different targets use different names.

Before running command, remember to configure your gcloud credentials. Add GOOGLE_APPLICATION_CREDENTIALS to your env variables.

./gradlew :beam-runners-google-cloud-dataflow-java-examples:preCommitLegacyWorker -PdataflowProject=<GcpProjectName> -Pproject=<GcpProjectName> -PgcpProject=<GcpProjectName> -PgcsTempRoot=<Gcs location in format: gs://..., no trailing slash> -PdataflowTempRoot=<Gcs location in format: gs://...> --scan

Running a User Defined Pipeline (example is on Java Direct Runner). 

If you want to run your own pipeline, and in the meanwhile change beam repo code for dev/testing purpose. Here is an example.

If it is just a simple runner like directRunner, all you need to do is to put your pipeline code under example folder, and then add following build target to the related build.gradle: 

There are also alternative choices, with slight difference:

1) Create a maven project. And use  ./gradlew -Ppublishing PublishToMavenLocal, to publish changed code to local repository. 

2) Make use of Integration tests, and make your user defined pipeline part of the integration test. 

Continue on error

--continue – this flag makes compileJava task to dump all found errors, not stop on first.