THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
To create a Spark project for IntelliJ, simply check out the repository and use the import functionality in IntelliJ to import Spark as a Maven project.:
- Download IntelliJ and install the Scala plug-in for IntelliJ.
- Go to "File -> Import Project", locate the spark source directory, and select "Maven Project".
- Click through to the profiles selection, and select the following profiles: yarn, scala-2.10, hadoop-2.4, hive-thriftserver, hive-0.13.1. Click through to create the project.
- At the top of the leftmost pane, make sure the "Project/Packages" selector is set to "Packages".
- Right click on any package and click “Open Module Settings” - you will be able to modify any of the modules here.
- A few of the modules need to be modified slightly from the default import.
- Add sources to the following modules: Under “Sources” tab add a source on the right.
- spark-hive: add v0.13.1/src/main/scala
- spark-hive-thriftserver v0.13.1/src/main/scala
- spark-repl: scala-2.10/src/main/scala and scala-2.10/src/test/scala
- For spark-yarn click “Add content root” and navigate in the filesystem to yarn/common directory of Spark
Eclipse
Eclipse can be used to develop and test Spark. The following configuration is known to work:
...