Page History

...

$ cd $SPARK_HOME/docs
$ jekyll serve --watch
$ sudo apt-get install linkchecker
$ linkchecker -r 2 http://localhost:4000 --no-status --no-warnings

Run License Audit Tool (not necessary for Spark 1.0)

Check whether all the source files have Apache headers.  Note that Spark REPL files and some pyspark files will have other license headers (as they are not Apache licensed) and should be ignored.

$ java -jar /path/to/apache-rat-0.10.jar --dir . --exclude *.md > rat_results.txt
$ vi rat_results.txt
$ # Look for source files that seem to have missing headers
$ cat rat_results.txt  | grep "???" | grep -e \.scala$ -e \.java$ -e \.py$ -e \.sh$
$ # Add missing headers if necessary

Create new CHANGES.txt File

The new CHANGES.txt can be generated using this script.

...