You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

This proposal emerged as a result of attempt to fix commit-validation  suite for the NetBeans distribution. Especially layers became broken over the time. The most painful area is per-MIME type registration underneath Editor folder.

MIME Inheritance

MIME registration have a sort of inheritance,  If we have a MIME type text/x-gradle+x-groovy, the Gradle+Groovy editor gets services from:

1. Editor/text/x-gradle+x-groovy{/S}
2. Editor/text/x-groovy{/S}
3. Editor{/S}

Where 'S' is an optional folder suffix. Note that services registered in subfolders do not mix, and they do not mix with 'generic' services in the base MIMEtype folder (no subfolder suffix) either.  The most generic registrations reside in Editor/{S }. MIME service registrations are usually processed in the order enumerated by the Lookup by editor and other services. So if one defines a position attribute for a registration .instance file, that file is ordered accordingly in the enumeration.

Important! The position order spans the whole hierarchy, not just one folder. So a generic service with position=100 will precede more specific service declared with position=200.

Service order

Suppose there's a service that many language supports define, such as CompletionProvider, or FoldManagerFactory. So each language has its own CompletionProviders. SOME languages depend on that CompletionProviders are defined in some specific order, mostly because "nice" ordering of suggestions. Examples are:

  • java (hint that scanning is in progress; java, javadoc, jpa suggestions)
  • jshell (java, history, shell commands)

Generic services (in Editors/) typically do not have any position/ordering information.  Most languages are just indifferent to the order. Maybe it's not important for the function, or the natural (declaration) order (textual order of appearance in the layer XML) is what the language implementation wants, especially if the registrations are written in the XML layer file. The important thing to remember is that some or mime-specific folders define order, but some do not.

So far so good.

Adding generic service

Now what happens, if someone comes with a bright idea to have a CompletionProvider that works for everyone ? Some cross-language feature like WordCompletion, SpellChecker or Language Server Protocol Client  suggestions ? According to the MIME inheritance rules, such provider is to be put in Editor/{S} - this is what the inheritance was invented for.

We have some (very small number) of languages that declare the order is important (their services are ordered in their MIMEtype folder). But with a generic registration, the order becomes important for everybody. Everyone has to define positions for that service type (items in the same subfolder, in all MIME types):

  • if the Editor/S/lsp.instance would have not defined an order, then commit-validation would fail on Editor/text/x-java/S, Editor/S not well-ordered: some items have positions, some do not, or
  • if the Editor/S/lsp.instance would have defined an order, then commit-validation would fail for all other MIME types except java that define services in S, which - up to now - were OK.

This does not seem much for the 'generic' implementor.

Indeed, we have escaped from this hell with WordCompletion, or SpellChecker since their providers is registered again and again for each individual MIME type. With Project Types, which also suffer from the same problem (see https://bz.apache.org/netbeans/show_bug.cgi?id=201893) - they decided to patch Filesystem Ordering with an exception instead see (https://github.com/apache/netbeans/blob/master/platform/openide.filesystems/src/org/openide/filesystems/Ordering.java#L161)

In the current situation, a generic service (in Editors/S ) cannot be safely registered - the committer risks that he will extend a contract (positions defined) from module Foo to all other modules.

Overall state

As it turns out, the most obtrusive change is the introduction of LSP CompletionProvider in the Editors/ (MIME type ""). Incidentally this is the service (CompletionProvider) that most languages see that 'natural declaration order' is sufficient. See, for example:
- Latte - https://github.com/apache/netbeans/blob/master/php/php.latte/src/org/netbeans/modules/php/latte/resources/layer.xml#L41
- Smarty - https://github.com/apache/netbeans/blob/master/php/php.smarty/src/org/netbeans/modules/php/smarty/resources/layer.xml#L63
- (Twig, ..)
- JSP - https://github.com/apache/netbeans/blob/master/enterprise/web.core.syntax/src/org/netbeans/modules/web/core/syntax/resources/layer.xml#L138
- HTML - https://github.com/apache/netbeans/blob/master/ide/html.editor/src/org/netbeans/modules/html/editor/resources/layer.xml#L149

Sidenote: it's obvious that we should do something more with inheritance or reuse: all the web templating language build upon HTML and add their own. For consistency, they *should* be able to inject all stuff defined for HTML, and define only the difference. But that's for a future Friday project.

During fixing commit-validation, in PR-2359 (https://github.com/apache/netbeans/pull/2359), I tried to introduce automatically generated positions, if the annotated element just does not care: this fixes the usual MIME use case: if the provider does not care, just define _some_ order, at the end (to remind: a special value of 0 means that the position is irrelevant, but should be put first.

It turned out that all those declaration order registrations (Latte, HTML, ...) broke and had to define an explicit order now, so they do not produce a warning in presence of auto-generated position attributes. Which is not entirely bad: Without an order explicitly set, with sufficient 'gaps', the MIME type is not extensible: If a XML (text/x-xml) defines two providers with no order, then it is impossible for JavaFX FMXL (text/x-fxml+xml) to insert a provider in between those two.

But for most languages,which (typically) do not provide services or boilerplate for others. this is just a complication.

Look at an experimental branch and its diff against master: https://github.com/apache/netbeans/compare/master...sdedic:origin/bugfix/commitvalidation-fixes2?expand=1

Summary of problem

  • there's NO WAY how to register a service in Editors/ (unspecified mimetype), that would pass current commit-validation
  • the unspecified mimetype ("") was specifically designed to define universal services, for all editors (without that, this is what the entire MIME inheritance is about)
  • once some obscure module eregisters a service in "" MIME type, he is likely to force all distribution modules to use positions in their registrations from that time onwards. Remember: if ANY mimetype uses positions in a specific MIME subfolder, all must use them (in that subfolder).

POSSIBLE SOLUTIONS:

Change test

DO NOT test for mix positions / no positions in Editors/* area under FileUtil.getConfigRoot(). This is the easiest, but we loose the assurance of some order of the registered services (which is IMHO too strict anyway, given the scenarios above)

Define positions (almost) everywhere

Sweep WHOLE DISTRIBTUTION, add positions where necessary. More or less easy; I already have changed most of them. What I personally do not like about this solution is the spread of contract of some module, just because of ANOTHER module makes generic registration.

Improve FileUtil.getOrder() contract

Change RUNTIME behaviour of FileUtil.getOrder(); see following proposal.

Filesystem Ordering improvement

The proposal somewhat complicates Filesystem Ordering definition, but would allow better reuse in MIME lookup, and hopefully in Projects lookups, too.

Currently we have one special value in Filesystem order, 0. This means "I don't care about exact order, sort me first - alphabetically". But, since some position is defined (0) in the folder, even if the registration basically does not care, it will imply that everyone has to provide an order. All other items (registrations) in that folder have to provide position attribute.

I propose to have values:

  • intvalue="0", defined as it is now: ordered first, order within 0 rack is alphabetical. Do not report duplicates. Compatilibity feature.
  • floatvalue="0.0": I don't care, sort last in the declaration order. Do not report duplicates.
  • floatvalue="-x.0" (no fractional part): IF order is defined, sort at the given position; check duplicates.

In addition neither of these values (note the change: even intvalue="0" !) will be treated as 'explicit position', so mix of position/undefined will not be reported, if the only defined positions will be 0es.

The MIME annotation processor would then by default generate nothing (again). LSP or other generic registrations cam use either 0.0, or -whatever.0, depending on whether order is or is not important. They could eventually use just intvalue="0", but that would also mean they would take precedence over more specific MIME registrations. Given how (implementation-wise) the MIME MultiFileSystem is composed:

  • position float="0.0" will favour the more specific declarations over the less specific ones, provided none of them has position defined;
  • declarations with positions defined will sort according to those positions, AFTER all intvalue="0" (compatibility) and BEFORE floatvalue="0.0" (new feature);

Benefits of this complicated setup are:

  • most modules can continue to write .instance files without any positions.
  • @MimeRegistration need not to specify or generate positions
  • wherever positions are defined, they are used to order the positioned items.
  • the hack for ProjectType service registry will be eliminated

Consistency check

The current consistency check is too strict - as outlined above, it basically prevents introduction of generic services to MIME lookup - only at expense of ordering the service folders in all MIME types in the the whole NetBeans distribution. But it is still good to ensure that the order is well-defined on individual levels of MIME registry, e.g. that a particular language either defines order for all service registrations, or for none of them. With the proposed changes above, generic services will be typically registered in a way that does not define order. So the test will primarily ensure that if the language defines order on services, the order is well-defined.


  • No labels