DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
1. After logging into the master node, download the YourKit Java Profiler for Linux from the YourKit downloads page(http://yourkit.com/download/index.jsp) ( (at the time of writing, the latest version is `yjp-12.0.5-linux.tar.bz2`(http://yourkit.com/download/yjp-12.0.5-linux.tar.bz2); you will need to substitute different paths if using a newer version). This file is pretty big (~100 MB) and YourKit downloads site is somewhat slow, so you may consider mirroring this file or including it on a custom AMI.
2. Untar this file somewhere (in `/root` in our case): `tar xvjf
| Code Block |
|---|
tar xvjf yjp-12.0.5-linux.tar. |
...
bz2
|
3. Copy the expanded YourKit files to each node using `copycopy-dir`: `~dir:
| Code Block |
|---|
~/spark-ec2/copy-dir /root/yjp-12.0. |
...
5
|
4. Configure the Spark JVMs to use the YourKit profiling agent by editing `~/spark/conf/spark-env.sh` and adding the lines
...
5. Copy the updated configuration to each node: `~
| Code Block |
|---|
~/spark-ec2/copy-dir ~/spark/conf/spark-env. |
...
sh
|
6. Restart your Spark cluster:
...
Please see the full YourKit documentation for the full list of profiler agent startup options(http://www.yourkit.com/docs/80/help/startup_options.jsp).