The following steps can be used to configure IntelliJ IDEA to build and test Daffodil:


  1. If not already installed, install the Scala plugin from JetBrains to add SBT and Scala support.

  2. With the Daffodil repository checked out, select

       File > New > Project from Existing Sources...

    and select the Daffodil repository directory.  Note that symlinks seem to cause failures to import, so use a absolute physical path to the daffodil repository directory.


  1. We want to configure the project to use SBT for configuration and building, so select:

       Import project from external model

    and select sbt.

  2. Set the following settings:

    1. Download Library sources: Checked

    2. Download sbt sources: Unchecked

    3. Use sbt shell for imports: Checked

    4. Use sbt shell for builds: Checked

    5. Allow overriding sbt version: Unchecked

    6. Global sbt settings > Maximum heap size: 6144 MB
      
  3. Click Finish.

  4. Exit IntelliJ and restart it, selecting the Daffodil project.

  5. An alert will display stating "sbt projects need to be imported". Select Enable Auto-Import.

  6. After the sync is complete in the Build tab, ensure that the project builds:

       Build > Build Project

  7. To add a configuration to run all JUnit tests, select:

       Run > Edit Configurations > + (plus sign) > JUnit

    and apply the following settings:

    1. Name: All Daffodil Tests

    2. Test Kind: Pattern

    3. Pattern: ^(?!.*TestCLI.*).*$

    4. Search for tests: In whole project
      
  8. Run tests via:

       Run > All Daffodil Tests

To setup to work with Daffodil/XML, you can start with the following:

  1. Go to File > Settings > Editor > File Types
  2. In the Recognized file types box, scroll down to XML , select it and under the Registered patterns box, click the + sign, and add *.dfdl.xsd. Repeat for *.tdml.

To be able to run CLI Tests, you either can try:

  1. IntelliJ Idea Ultimate ensuring the Working Directory is the project root. You can get a free license
    1. if you're an apache committer: https://www.jetbrains.com/community/opensource/#partner
    2. if you're a student/teacher: https://www.jetbrains.com/community/education/#students
    3. if you're part of an open source project: https://www.jetbrains.com/community/opensource/#support
  2. IntelliJ Idea Community as described in this comment (works for some, but not all).

The default formatting settings for Scala must be changed to match our code style too.  You must explicitly check two options that are disabled by default (File→Settings, Editor Scala, Spaces tab):

  • Import braces
  • Around '@' in pattern bindings



  • No labels

7 Comments

  1. Mike Beckerle

    When I tried to setup a large project, IDEA started complaining about inotify handles or some such.

    I had to follow the instructions here: https://confluence.jetbrains.com/display/IDEADEV/Inotify+Watches+Limit

  2. Mike Beckerle

    I discovered that having JAVA_OPTS with

    "-Dsun.io.serialization.extendedDebugInfo=true"

    increases the stack space needs substantially to the point where IDEA won't run the EDIFACT schema unit tests without enlarging the stack. It also slows things down significantly. 

  3. Mike Beckerle

    Some of IDEA's scala settings are not consistent with our formatting standard.

    I had to check File→Settings select Editor Scala. On the Spaces tab, check boxes for

    • "Import braces",
    • "Around '@' in pattern bindings"

  4. Mike Beckerle

    To be able to run the CLI tests within IDEA, I had to edit the configuration to make it look more ordinary, with tests in src/test instead of src/it.

    It's a pain in the neck because you don't want to commit these changes.

    These steps:

    1. edit build.sbt and change "it→test" to "test→test"
    2. cd daffodil-cli/src
    3. ln -s it test # creates a symlink named 'test' to the 'it' directory
    4. In project panel right click daffodil, and 'reload from disk'
    5. In IDEA in the sbt panel (upper right side to open) right-click daffodil and choose 'reimport sbt project'

    Outside of IDEA at the command line:

    • sbt stage

    Now back in IDEA you can run a CLI test interactively. However, it is shelling out to an external executable, so you can't debug interactively.

  5. Mike Beckerle

    I have found IDEA is very sensitive to symlinks vs. regular paths. I find when creating a new project, if the path I give to the project has a symlink in it, the import often fails. If I give the absolute path without symlinks it works fine.

  6. Mike Beckerle

    Turns out some of the nice electric completion and such for IDEA happens due to this file I found in my ~/.idea/ directory: IDEA-xml-catalog.xml

    Contains this. If daffodil libraries (esp the daffodil-lib library jar) are on classpath, as they will be for DFDL schema projects that use Daffodil, then this will allow IntelliJ IDEA to find the XML Schemas for DFDL file, and TDML files. 

    <catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog">
  <uri name="http://www.ibm.com/xmlns/dfdl/testData"
       uri="org/apache/daffodil/xsd/tdml.xsd" />
  <uri name="http://www.ogf.org/dfdl/dfdl-1.0/"
       uri="org/apache/daffodil/xsd/DFDL_part3_model.xsd" />
  <uri name="http://www.ogf.org/dfdl/dfdl-1.0/extensions"
       uri="org/apache/daffodil/xsd/dfdlx.xsd" />
  <uri name="http://www.w3.org/2001/XMLSchema"
       uri="org/apache/daffodil/xsd/XMLSchema_for_DFDL.xsd" />
  <uri name="urn:ogf:dfdl:2013:imp:daffodil.apache.org:2018:ext"
       uri="org/apache/daffodil/xsd/dafext.xsd" />
  <uri name="urn:ogf:dfdl:2013:imp:daffodil.apache.org:2018:int"
       uri="org/apache/daffodil/xsd/dafint.xsd" />
</catalog>

    Note that this is getting the corresponding schemas from IntelliJ IDEA's classpath, which is not the same as any given project's classpath. E.g., if you are editing these XSD schema files, then the IDEA may not see and take up your changes. In that case you may need to use absolute file paths in these entries, and even then forcing it to re-read the files may require restarting IDEA. 

  7. Mike Beckerle

    A further enhancement to the IntelliJ IDEA electric XSD, DFDL, TDML support:

    If you order the xmlns declarations at the top-level element, so that the default namespace declaration appears first, then when IDEA suggests things for you, it will suggest them without the namespace prefix.  Its suggestions use the order of xmlns bindings in the root element. 

    So if you use the preferred DFDL schema and TDML styles which use a default namespace, then setup DFDL schemas like this with the default namespace first.

    <schema  xmlns="http://www.w3.org/2001/XMLSchema"
  xmlns:xs="http://www.w3.org/2001/XMLSchema"
  ....

    And for TDML files:

    <testSuite
  suiteName="NameDOB"
  xmlns="http://www.ibm.com/xmlns/dfdl/testData"
  xmlns:tdml="http://www.ibm.com/xmlns/dfdl/testData"
  xmlns:xs="http://www.w3.org/2001/XMLSchema" 
  xmlns:fn="http://www.w3.org/2005/xpath-functions"
  xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/" 
  ...