Page History

Historically NetBeans rely on (nb-)javac to provide javac  for Java editing features, i.e., parsing and lexing, for features such as syntax coloring, code completion, refactorings, etcand the like.This has some positive aspects, but also some downsides. Originally nb-javac  was maintained in a separate repository: hg.netbeans.org/main/nb-javac - Since version 15, it is hosted by Arvind's team on GitHub: https://github.com/oracle/nb-javac - "nb-javac" consists of two libraries ( nb-javac-api.jar  and nb-javac-impl.jar ) located in "extra" cluster, see: Overview: NetBeans Structure.

Pros:

• Adopting latest Java language features is simple(r)
• Errors, hints, warnings in the editor match exactly the command line build
• Close co-operation with JDK language team

Cons:

• Distributing GPLv2+CPEx licensed component in Apache software is hard

  • it would be way easier to use plain `javac` plain javac  from a JDK

  • nb-javac is   has to be downloaded by end-user on demand via autoupdate

• On demand download is problematic

  • user needs internet connection 

  • download server needs to be on (nb-javac isn't yet on Maven central)

  • e.g. sometimes download fails

• Testing matrix is complicated

  • each supported JDK needs to be tested twice - with nb-javac  and without nb-javac 

• Every bug/problem one needs to know whether nb-javac  was or wasn't in use

  • Recent version nb-javac-15  isn't really stable

• nb-javac  is a fork of JDK's javac

  • nobody likes forks

  • ironically Arvind's team is part of JDK organization - e.g. it maintains own fork of JDK's `javac`

• Upstream JDK javac hasn't been IDE ready up to JDK15
  • see below for details
• Without NbJavac one needs to run NetBeans on latest JDK to get latest language features
  • Serious disadvantage compared to competitors IDEs
  • They have their own Java parsers and support latest features on any JDK

What is nb-javac?

"nb-javac" consists of two libraries (nb-javac-api.jar and nb-javac-impl.jar) that provide a NetBeans-specific fork of the Java JDK Java compiler.

What exactly does nb-javac do?

NetBeans uses nb-javac for the Java Editor, i.e., parsing and lexing for features such as syntax coloring, code completion, and the like.

Where is nb-javac found in the NetBeans Mercurial repo?

Originally nb-javac was maintained in a separate repository: hg.netbeans.org/main/nb-javac - Since version 15, it is hosted on GitHub: https://github.com/oracle/nb-javac

Where is nb-javac found in the NetBeans installation directory?

In the "java" folder, provided by the "java" cluster, see: Overview: NetBeans Structure

Is nb-javac going to be Apache licensed?

No. nb-javac is not part of Oracle's donation of NetBeans to Apache.

How can nb-javac be distributed, given that it will not be Apache licensed?

...

is

...

nb-javac

...

?

...

What does nb-javac do that's different to vanilla javac?

A few years back, the NetBeans team wrote a page describing what nb-javac does in addition to/differently to vanilla javac. Some of the things listed above are not part of nb-javac anymore (e.g., cancelling and some part of the error recovery paragraph) but it still gives a reasonable overview and is good to get an idea what kinds of things nb-javac does. Here is the copy ready to be made up to date:

Recompleting Symbols from Sources

Consider the following usecase: let there be two classes, A and B available both in source code and (up-to-date) class files. These classes are interdependent (each refers to the other). Let there be a refactoring, that needs to work over both these files. The refactoring needs Trees (to access method bodies, to get offsets, etc), and these trees need to be attributed.

...

The ability to recomplete a Symbol from source file is one of the most important part of the NetBeans fork/patch.

Stable annonymous innerclass numbers

When a Scope is created, the corresponding Tree is duplicated and attributed. This may lead into incorrect Symbols created for anonymous innerclasses. It is necessary to ensure that the anonymous innerclass numbers will match the numbers that would be produced by a batch compiler. This needs to be ensured even in case when the Scope is created before the tree is attributed.

Error Recovery

While editing, the code in the editor contains compilation errors almost all the time. It is therefore unacceptable to loose e.g. code completion only because of a missing semicolon, or because of an (unrelated) unresolvable symbol.

...

• JavacParser:
  • a variable declaration tree is produced even for cases where variable declaration is not allowed, e.g. in then/else section of if. The tree is wrapped in an erroneous tree and an error is reported. Trees produced before this change were too confusing for some hints (NB bug #192561). Relates to parseStatement and parseBlockStatement(s).
  • parseCompilationUnit is modified to handle multiple package clauses
• Attr:
  • in "return <expr>" <expr> is always attributed, even for initializers and methods with return type void
  • the last-resort symbol filler PostAttrAnalyzer is modified to dive into erroneous trees (NB bug #152334)
• Annotate:
  • modified to always attribute the annotation's attribute's values, even if the annotation itself is unresolvable (i.e. "@Undefined(@Another)" will attribute also @Another).

Cancellability

NetBeans need to be able to stop javac processing even inside one phase (parse/member enter/attribute). This is used e.g. when the user types into the editor, and the current instance of javac is already processing an obsolette source code. Then, there is no point in wasting time and memory in continuing the processing of the file.

Inferring Binary Names

Consider file A.java, containing classes B and C, and file D.java, referencing B and C. If file D.java is being parsed (and class file for B and C do not exist yet), the javac is currently unable to locate file A.java. Yet, the IDE may have the information about the content of the A.java file. So, the problem is how to pass this information into the javac. ClassNamesForFileOraculum is currently used for this.

Support for Reparsing Method Bodies

If the user changes are contained only inside one method body, it is desirable to reparse only the body of the one method. This leads into faster reparse times and less garbage on the heap. The NetBeans' fork/patch contains support for reparsing methods bodies.

Repair

In the NetBeans' fork/patch, there is a new phase, Repair, running after Flow. For source code with compilation errors, this phase converts the "uncompilable" trees into "compilable" trees.

...

• error types (stripped from the trees during repair to obtain valid trees)
• parameters names, without depending on debug information
• annotations with RetentionPolicy.SOURCE

Annotation Processing

In vanilla javac, new Symbols are created for classes/methods/fields in the sources for each annotation processing round. This is rewritten in the NetBeans' fork/patch to use the symbol recompleting. Symbols, once created, are used in all rounds on annotation processing and also for the final compilation. Annotation processing is also supported when completing symbols from sources. Exceptions thrown by annotation processors are logged, but do not stop the compilation.

...

IDE Mode

A special option, "ideMode", has been introduced by the NetBean's fork/patch to improve javac behavior in the following cases:

• when there is no java.lang package (i.e. no platform), vanilla javac stops with an error. In IDE mode, the compiler handles the situation more gracefully. - !May not be needed anymore.
• when loading 1.5+ classfile with sourcelevel <=1.4, vanilla javac throws away any 1.5 information stored in the classfile. In IDE mode, this information is preserved.

Miscallenous

• small extensions to allow parsing and attributing a "standalone" statement, expression, etc. in the given context.
• small extensions to JavacTaskImpl to allow per-file per-phase parsing of multiple sources in the same instance of javac
• access restrictions loosened for a lot methods/classes (private->protected, etc.)
• inlineTags cache for ParamTagImpl and ThrowsTagImpl
• string folding is disabled (by "disableStringFolding" option) to so that concatenated strings are represented by full trees
• ToolProvider loads classes from context classloader
• TreeInfo.symbolFor works all trees that refer to a symbol

What is the reason that nb-javac is a fork and not part of the javac main repository?

1. The pace at which javac/JDK moves and at which NetBeans moves is simply different. E.g., imagine a refactoring breaking a user's code because of error recovery misbehaving. Once the problem is found it may be fixed in nb-javac in hours or days. Fixing it in javac can easily take weeks or even months (and much more effort in testing and review). 
   
   
2. Even some changes that from the NetBeans point may look innocent may be hard to get into vanilla javac. For example, when javac detects a syntax error, it normally ends without getting to a semantic analysis. This does not work for an IDE (noone probably wants to lose almost all features just because a semicolon is missing). But it may be difficult to push through a fix to semantic analysis for a syntactically broken code (sometimes this passed before, other times, there was a pushback on such changes, depending on the nature of the change needed). 
   
   
3. Some of the changes in nb-javac may be very hard to get in unless it has another user than just NetBeans.

Taking a step back, what are the potential alternative Java parsing libraries to support the optional module for Java in NetBeans?

1. Write a new library/compiler. Not quite realistic.
    
2. Use some other than javac (like from Eclipse). Not sure what would be the differentiator between Eclipse and NetBeans? Would result in incompatible changes and probably almost a complete re-write of the existing Java support in NetBeans (unless the Java support itself would be taken from Eclipse as well, which aside from everything else would be a lot of rework too). 
    
3. Use a vanilla (unpatched) javac from the current runtime JDK. Besides other complexities, one would need to run on JDK 9 to get the possibility of JDK 9 support (and on, e.g., Valhalla to get the possibility of having Valhalla support), which might possibly lead to workarounds for differences/bugs between JDK versions. Plus, we might run into trouble with strong encapsulation. 
    
4. Use a vanilla (unpatched) javac corresponding to a particular revision from the OpenJDK, but let NetBeans have a firm (binary) copy against which it would run, regardless of which runtime JDK would be used. Still a significant endeavour. 
    
5. Keep nb-javac as it is now. Certainly the least amount of work and complexity.

...