Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Cleaned up many references to Ant.

...

Note
titleUse Gradle for Solr 9 and Ant for Solr 8 and earlier

Solr has transitioned from Ant to Gradle to  as it's build system starting with Solr 9. See: Building Solr with Gradle.

Trunk/Solr9 can be built only with Gradle. Previous versions of Solr, including all 8x versions can only be built with Ant. See: Building Solr with Ant.

...

  • All public classes and methods should have informative Javadoc comments.
  • Code should be formatted according to Sun's conventions, with some exceptions:
    • indent two spaces per level, not four.
    • lines can be greater than 80 chars long, 132 is a common limit. Try to be reasonable for very long lines.
  • Contributions should pass existing unit tests.
  • New unit tests should be provided to demonstrate bugs and fixes (http://www.junit.org).

Notes for Eclipse

...

Info
titleUse Gradle for Solr 9, Ant for 8x and earlier

For Solr 9, gradlew eclipse or gradlew idea can be used to set up the project, simply execute that command and open the project.

For Solr 8 and earlier use ant eclipse or ant idea

Generating a patch

A "patch file" is the conventional contribution mechanism. It bundles up everything that is being added, removed, or changed in your contribution. You can instead do a GitHub pull request, which is documented elsewhere on this page; nevertheless most of this section is still applicable.

...

Please make sure that all unit tests succeed before constructing your patch.

No Format

> cd solr-trunk

For Solr 9+, 
> gradlew clean test

For Solr 8 and earlier
> ant clean test

After a while, if you see a success or failure message

No Format

BUILD SUCCESSFUL

all is ok. but if you see

No Format

BUILD FAILED

please, read carefully the errors messages and check your code. If the test fails you may want to repeatedly rerun a single test as you debug and sort out any problems. In which case you could run

No Format

> ant -Dtestcase=TestXXX test

Where "TestXXX" is the name of the particular Junit test you want to run. NOTE: specifying -Dtestcase= must be done at least at the <home>/lucene or <home>/solr level. Executing this at the root level will not run tests.

Frequently failing Tests

There are some tests that fail sometimes on some systems, but run on Jenkins fine. It's always a good idea to be sure you can run "ant test" successfully before you start making code changes. Or keep an un-changed version of the code around to see if your changes are really to blame.

One of the great things about Open Source is so many people run the tests on so many different systems. Occasionally you'll be the lucky person who has the system that wins the prize by having the environment that exposes a new failure mode, see the discussion at SOLR-3846 for an example.

If you do find one of these, here's what you should do:

...

Solr testing makes extensive use of randomization. Each test starts with a "seed" for the random number generator, allowing repeatability. We had one test, for instance, that only failed when the locale was set to a particular locale. Re-using the seed reproduces these kinds of cases.

You'll find a "reproduce with..." message either on the screen or in the output that gives the exact command necessary

Carefully read the errors messages and check your code. If the test fails you may want to repeatedly rerun a single test as you debug and sort out any problems. In which case you could run the "reproduce with" command in the output.

Frequently failing Tests

There are some tests that fail sometimes on some systems, but run on Jenkins fine. It's always a good idea to be sure you can run the full test suite successfully before you start making code changes. Or keep an un-changed version of the code around to see if your changes are really to blame.

One of the great things about Open Source is so many people run the tests on so many different systems. Occasionally you'll be the lucky person who has the system that wins the prize by having the environment that exposes a new failure mode, see the discussion at SOLR-3846 for an example.

If you do find one of these, here's what you should do:

  • If tests continue to fail, ask on the dev list if anyone else has seen the issue. This is the case where having the un-changed code helps. If the tests fail on both the changed and un-changed versions, discuss on the dev list whether the test can be annotated as a 'badapples' test or notshould be disabled.
  • If tests fail with your changes but not on un-altered code, well, you need to understand why. By far the most frequent reason is a bug or unintended side-effect of your code, but occasionally it's a test that needs modification. Again, the dev list is a good place to discuss this.
  • Be very cautious about adding anything like @Ignore to a test. This is generally the wrong thing to do unless you get some consensus, and it'll surely generate "spirited debate".
  • Of course any effort you want to make towards tracking down the reason a test fails in your particular environment is greatly appreciated!

Pre-commit check

Before constructing considering your patchwork done, please you should run the top-level pre-commit check, which finds problems like tabs and @author tags in source files, broken links in javadocs, files not controlled by Git (a.k.a. "unversioned files"), unused imports etc.

To run the pre-commit checks from ant, run the following from the top-level directory – the directory containing lucene/ and solr/ – in your working copy:

No Format

ant precommit
No Format
For Solr 9+
At the top level, run "gradlew check". NOTE: this runs checks and all tests. To just run the checks, run "gradlew check -x test"

For Solr 8 and earlier
At the top level, run "ant precommit"

Creating the Creating the patch file

Check to see what files you have modified with:

No Format

git status

Add any new files with:

No Format
git add src/.../MyNewClass.java
or, if all the changed files are part of the code changes,
git add .

Git's "add" command only modifies your local copy, so it does not require commit permissions. By using "git add", your entire contribution can be included in a single patch file, without needing to submit a separate set of "new" files.

...

Edit the CHANGES.txt file, adding a description of your change, including the bug number it fixes.

In order to create a patch, just typeTo submit your changes for review:

No Format

git format-patch origin/master > Git Pull Requests (PRs) are generally preferred.


If you prefer to attach a patch to the JIRA, especially for very small changes, "git format-patch origin/master > SOLR-NNNNN.patch"

This will report all modifications done on Solr sources on your local disk and save them into the SOLR-NNNNN.patch file. Read the patch file. Make sure it includes ONLY the modifications required to fix a single issue.

You should quickly review your PR or patch to see if it's what you expect.

If you prefer patches, Note the SOLR-NNN.patch patch file name. Please use this naming pattern when creating patches for uploading to JIRA. Once you create a new JIRA issue, note its name and use that name when naming your patch file. For example, if you are creating a patch for a JIRA issue named SOLR-12345, then name your patch filename SOLR-12345.patch. If you are creating a new version of an existing patch, use the existing patch's file name, the JIRA system will automatically "do the right thing".

Please do not:

  • reformat code unrelated to the bug being fixed: formatting changes should be separate patches/commits.
  • comment out code that is now obsolete: just remove it.
  • insert comments around each change, marking the change: folks can use git to figure out what's changed and by whom.
  • make things public which are not required by end users.
  • Combine multiple issues into a single patch, especially if they are unrelated or only loosely related. This is true even if the changes affect the same files. In some rare cases it is warranted, but for the most part it makes it harder for committers to evaluate the patch.

...

  • try to adhere to the coding style of files you edit;
  • comment code whose function or rationale is not obvious;
  • update documentation (e.g., package.html files, this wiki, etc.)
  • try to provide a unit test that shows a bug was indeed fixed or the new functionality truly works

Contributing your work

Patches PRs and patches should be attached to a bug report in Jira. If you are revising an existing patch, please re-use the exact same name as the previous attachment. JIRA will automatically "gray out" the old patch and clearly mark your newly uploaded patch file as the latest (it'll be the colored link). You'll see multiple copies of your patch if you've named them identically, and this is preferred as it preserves the history of the patch which can come in handy. Since the most recent one is the only one not gray, we always know which one to use.

...

  • Eclipse (Eclipse codestyle).
  • IntelliJ (IntelliJ codestyle) NOTE: there is no need to install this file separately if you execute "ant idea" or "gradlew idea", it is done for you. If you do install it, you'll have to tweak the name to select in in IntelliJ as it's currently anonymous (all the more reason to use the ant target, see the IntelliJ instructions).

(warning) Solr1.3 If you are using eclipse to follow trunk (leading up to the 1.3 release) eclipse will give several errors about not resolving components in the solrj library. This will appear in the org.apache.solr.handler.component package relating to distributed search (sharedrequest.java ...etc) The solution is to compile the solrj library via the dist-solrj target and add them to your eclipse build path. After running the dist-solrj target look in dist/solrj-lib and add apache-solr-solrj-1.3-dev.jar and commons-httpclient-3.1.jar to your buildpath.

Development Environment Tips

Here is a guide for setting up Eclipse, IntelliJ and Netbeans dev environments:

Follow the instructions above to fetch the combined Lucene and SOLR trunk. For the remainder of this document, the installation path is assumed to be ~/apache/trunk/lucene and ~/apache/trunk/solr. NOTE: I'm installing on a Macbook, so this is a *nix style file system etc. These instructions should work for windows as well, but if you try to use them in that environment, feel free to update this page with anything you uncover.

Before fiddling with the IDE, I'd strongly recommend you get the tests to run from the shell. This will insure that your machine has the proper setup for the IDEs to magically find what's necessary. See the instructions above. Hint: Issue 'ant clean test' in the SOLR and Lucene directories and look for "BUILD SUCCESSFUL" minutes later.

Setting things up is actually very smooth when it's smooth, especially if the tests have run <G>.

Development Environment Tips

Here is a guide for setting up Eclipse, IntelliJ and Netbeans dev environments:

Follow the instructions above to fetch the combined Lucene and SOLR trunk. For the remainder of this document, the installation path is assumed to be ~/apache/trunk/lucene and ~/apache/trunk/solr. NOTE: I'm installing on a Macbook, so this is a *nix style file system etc. These instructions should work for windows as well, but if you try to use them in that environment, feel free to update this page with anything you uncover.

Before fiddling with the IDE, I'd strongly recommend you get the tests to run from the shell. This will insure that your machine has the proper setup for the IDEs to magically find what's necessary. See the instructions above. Hint: Issue 'gradlew check' or 'ant clean test' in the SOLR and Lucene directories and look for "BUILD SUCCESSFUL" minutes later.

Setting things up is actually very smooth when it's smooth, especially if the tests have run <G>.

DO DO NOT BE SURPRISED IF SOME TESTS FAIL IN THE IDE. There are some anomalies when running Junit tests for these projects in an IDE. Some of them are already cleaned up, but others may still fail when run in an IDE. The definitive case for whether a test fails or not is running it as an a Gradle or Ant tasktest.

Eclipse

See wiki page on configuring Eclipse.

...

Note: Fixing these issues may require asking questions on the developer list to figure out what they mean - there is no guarantee that any of these will be either quick or easy.

Parsers and JFlex

YOU PROBABLY DON'T HAVE TO DO THIS but if you do for some reason want to regenerate the .java from the .jflex files, here's the cheat-sheet. This is a separate step that people who actually like parsers sometimes have to execute to reflect changes in various grammars.

  • Check out the most recent jflex branch. This is important since what Lucene uses is based on trunk rather than a released version (thanks for tutoring me Uwe!). Do this by executing: svn co https://jflex.svn.sourceforge.net/svnroot/jflex/trunk jflex
  • cd jflex
  • mvn install (NOTE: I've seen the tests fail in this step, doesn't seem to matter)
  • cd <solr_home>/lucene/analysis/common
  • export ANT_OPTS="-Xmx1G -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=1G" (1)
  • ant jflex

Generated files

There are several files that are generated, for instance several of the tokenizers implement a scanner that is defined in a ".jflex" file. It is rarely necessary to modify the ".jflex" file and almost always a mistake to hand-edit the ".java" file generated from it. If you see any message at the top of a file you're working with like "/* The following code was generated by JFlex 1.7.0 */" please discuss why you want to change it on the development list before proceeding(1) I was getting OOM errors when running "ant jflex", and got a tip to do this, which made that problem go away. You may not have to be so generous with memory, but this worked... (Thanks Steve!).

One final note

As always, there are gremlins out there. If you find problems with the information here, and especially if you subsequently find solutions to the problems you find, please either write to the solr-user mailing list or update this page directly so others can benefit.