Page History

An explanation of the key aspects to the place and purpose of nb-javac. 

What is nb-javac?

"nb-javac" consists of two libraries (nb-javac-api.jar and nb-javac-impl.jar) that provide a NetBeans-specific fork of the Java JDK Java compiler.

What exactly does nb-javac do?

NetBeans uses nb-javac for the Java EditorHistorically NetBeans project relies on nb-javac  for Java editing features, i.e., parsing and lexing, for features such as syntax coloring, code completion, refactorings, and the like.

Where is nb-javac found in the NetBeans Mercurial repo?

In a separate repository: hg.netbeans.org/main/nb-javac

Where is nb-javac found in the NetBeans installation directory?

In the "java" folder, provided by the "java" cluster, see: Overview: NetBeans Structure

Is nb-javac going to be Apache licensed?

No. nb-javac is not part of Oracle's donation of NetBeans to Apache.

How can nb-javac be distributed, given that it will not be Apache licensed?

Of course of paramount importance is that the user encounters no problems at all and that NetBeans as always works out of the box. The most obvious mechanism to installing nb-javac is to do it in the same way as done for JUnit (before JUnit licensing changed and it became a standard part of NetBeans), i.e., during installation a question can be asked "Which languages would you like to work with?" and if Java is chosen nb-javac will be downloaded and installed:

wiki.apidesign.org/wiki/AutoUpdate

What does nb-javac do that's different to vanilla javac?

A few years back, the NetBeans team wrote a page describing what nb-javac does in addition to/differently to vanilla javac:

http://wiki.netbeans.org/JavacDiff

Looking at the page now, some of the things listed above are not part of nb-javac anymore (e.g., cancelling and some part of the error recovery paragraph) but it still gives a reasonable overview and is good to get an idea what kinds of things nb-javac does. 

What is the reason that nb-javac is a fork and not part of the javac main repository?

...

NetBeans 13 uses Javac from JDK 17!

• Approved

  Jira
  server ASF JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key LEGAL-563

• Repository with automatically generated nb-javac
• PR-3206 integrates the below proposed solution into NetBeans 13.
• PR-1 makes sure only Classpath exception  covered files are used in the nb-javac  generated binary
• PR-3251 includes the here-in proposed Javac in all NetBeans complementary binaries
• PR-2783 allows compilation against JDK-17 Javac API - removes need for (reflection & co.) hacks

Javac from JDK is Great!

Relying on Javac has some positive aspects, but also some downsides.

Pros:

• Adopting latest Java language features is simple(r)
• Errors, hints, warnings in the editor match exactly the command line build
• Close co-operation with JDK language team

Cons:

• On demand download (of newest javac on old JDKs) is problematic

  • user needs internet connection 

  • download server needs to be on (finally nb-javac is at least hosted on Maven central)

  • e.g. sometimes download fails

• Code is complicated
  • Supporting multiple releases of javac (from different JDKs) complicates NetBeans code
  • NetBeans code in java.source  & related modules if full of reflection
  • NetBeans is using internal API of javac (Trees API) and it changes incompatibly with every release

• Testing matrix is complicated

  • each supported JDK needs to be tested twice - with nb-javac  and without nb-javac 

  • every JDKs javac is a bit different

• 
  

• Every bug/problem one needs to know whether nb-javac  was or wasn't in use

  • Recent version nb-javac-15  isn't really stable

• Old nb-javac  is a fork of JDK's javac

  • nobody likes forks

  • ironically Arvind's team is part of JDK organization - e.g. it maintains own fork of JDK's javac 

Eliminating the need for nb-javac

Clearly there are numerous drawbacks and Apache NetBeans needs a way out. Let's get rid of nb-javac  as we know it. Let's replace it with JDK's own javac ! However there are some problems...

•  javac  in JDK15 isn't good enough
• compile on save doesn't work
• re-compilation of a single method doesn't work
• runs out of memory more often than `nb-javac`.

Before NetBeans can really get rid of nb-javac , the `javac` in JDK is needs to be good enough.

Using JDK 17 javac Instead

Let's now assume JDK17 offers good enough javac , now NetBeans project can suggest/require people to use JDK17 to run Apache NetBeans IDE

• not a big problem, JDK17 is LTS, but then?
• if people wanted to use language features of JDK19, they'd have to run on 19!
• that's not what competition does - they support latest language features running on JDK11 LTS or even JDK8 LTS

Requiring the latest JDK to execute the IDE is serious disadvantage compared to competitors IDEs, but possibly the story may end here and it might even be a good enough story for Apache NetBeans IDE. However...

Automatically Generating nb-javac

However, I don't find the restriction of latest JDK satisfying. It is not good enough story yet. There are parties that want to run on the IDE on some Java LTS version and still support the latest Java features. To address their needs let's take JDK17's javac  and let run it on JDK8! Of course, there are issues:

• latest javac  is written in the language syntax of modern Java
  • such syntax cannot be compiled to JDK8 bytecode with `javac`
• latest javac  is using APIs not available on JDK8
  • one needs to rewrite these calls to some older APIs
  • the behavior needs to be tested to remain the same

The great revelation is that both these problems can be solved with existing Apache NetBeans tools! Rather than maintaining manual patches like nb-javac  does, let's write advanced refactoring rules and apply them automatically. For example Optional.isEmpty()  method has been added in JDK11. Let's add following rule:

$1.isEmpty() :: $1 instanceof java.util.Optional
=>
!$1.isPresent()
;;

That automatically rewrites all occurrences of optional.isEmpty()  to !optional.isPresent()  and that is going to compile on JDK8. Few more (~30) rules like this and the javac  is almost ready to run on JDK8! Run few tests to verify the behavior remains the same after the automatic transformation and that's all. People can use Apache NetBeans IDE with javac  from the latest JDK or they can use the automatic port of the same code running on JDK8. Ideally the behavior shall be identical. No more questions: Are you using nb-javac or not? No more duplicated testing matrix.

Taking a step back, what are the potential alternative Java parsing libraries to support the optional module for Java in NetBeans?

1. Write a new library/compiler. Not quite realistic.
    
2. Use some other than javac (like from Eclipse). Not sure what would be the differentiator between Eclipse and NetBeans? Would result in incompatible changes and probably almost a complete re-write of the existing Java support in NetBeans (unless the Java support itself would be taken from Eclipse as well, which aside from everything else would be a lot of rework too). 
    
3. Use a vanilla (unpatched) javac from the current runtime JDK. Besides other complexities, one would need to run on JDK 9 to get the possibility of JDK 9 support (and on, e.g., Valhalla to get the possibility of having Valhalla support), which might possibly lead to workarounds for differences/bugs between JDK versions. Plus, we might run into trouble with strong encapsulation. 
    
4. Use a vanilla (unpatched) javac corresponding to a particular revision from the OpenJDK, but let NetBeans have a firm (binary) copy against which it would run, regardless of which runtime JDK would be used. Still a significant endeavour. 
    
5. Keep nb-javac as it is now. Certainly the least amount of work and complexity.

Show me the patch!

hg clone http://hg.netbeans.org/main/nb-javac/
cd nb-javac
hg diff -r 8e9e1a2373a4 -r 061355d195b

Note: It is huge (this full patch has more that 35000 lines, though that includes additional tests, additional project metadata, etc). The wiki page above gives a better overview of what the patch does.

There are about 10k LLOC with additional tests and about 10k LLOC of modifications with the main purpose of exposing more internals and better error recovery.