Page History

...

Eclipse can be used to develop and test Spark. The following configuration is known to work:

• Eclipse Juno
• Scala IDE v 34.0.3
• Scala Test

Scala IDE can be installed using Help | Eclipse Marketplace... and search for Scala IDE. Remember to include Scala Test as a Scala IDE plugin. To install Scala Test after installing Scala IDE, follow these steps:

• Select Help | Install New Software
• Select http://download.scala-ide.org... in the "Work with" combo box
• Expand Scala IDE plugins, select ScalaTest for Scala IDE and install

The easiest way is to download the Scala IDE bundle from the Scala IDE download page. It comes pre-installed with ScalaTest. Alternatively, use the Scala IDE update site or Eclipse Marketplace.

SBT can create Eclipse .project and .classpath filesSBT can create Eclipse .project and .classpath files. To create these files for each Spark sub project, use this command:

...

To import a specific project, e.g. spark-core, select File | Import | Existing Projects into Workspace. Do not select "Copy projects into workspace". Importing all Spark sub projects at once is not recommended.

If you want to develop on Scala 2.10 you need to configure a Scala installation for the exact Scala version that’s used to compile Spark. At the time of this writing that is Scala 2.10.4. Since Scala IDE bundles the latest versions (2.10.5 and 2.11.6 at this point), you need do add one  in Eclipse Preferences -> Scala -> Installations by pointing to the lib/ directory of your Scala 2.10.4 distribution. Once this is done, select all Spark projects and right-click, choose Scala -> Set Scala Installation and point to the 2.10.4 installation. This should clear all errors about invalid cross-compiled libraries. A clean build should succeed now.

ScalaTest can execute unit tests by right clicking a source file and selecting Run As | Scala Test.

...