Status

Current state: Under Discussion

Discussion thread: https://mail-archives.apache.org/mod_mbox/lucene-dev/202004.mbox/browser

JIRA: Unable to render Jira issues macro, execution error.

Released: 9.0.0

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast). Confluence supports inline comments that can also be used.

Motivation

1. Improve overall security by removing EOL code from the code base
2. Preserve functionality by providing an updated Admin UI
3. Make Solr more approachable by both users and UI devs, while hopefully making it easier to get started with Solr today.

Public Interfaces

Same interface, new look. Existing Solr Admin UI would be removed by the time we are preparing to release 9.0 because we will have reached feature parity and have integrated the application into the standard pipelines for starting and deploying Solr. This could happen before we reach 9.0 as well and it may behoove the community at large to strip out the old UI as soon as 

Proposed Changes

Build an Admin UI with the latest version of Angular (9.2.1). Our current version is about five years old.

The new UI will use a module loader and npm to start the app.

The new UI will convert the language to TypeScript as a strongly-typed language fits the Java programmer's paradigm more.

Many of the existing features will improve in terms of accessibility and capabilities. For example, logs will be sortable by every field, searchable, and paginated in the end state.The front page updates will make it easier to discern issues for new users or regular users of the Admin UI.

Development

The proposed run time will be be delivered over the course of 2 phases, with the option of a third phase if we can find the time to collaborate but it is by no means required because it adds lots of work. Sandbox, as a Jetty-issued command, and then possibly as a package manager if I can collaborate with the package management team (currently not an option). 

As of today, here are the proposed phases:

1) In the sandbox phase, where we are today, users can run Solr as they normally would. If they would like to try the new UI, they need to navigate to the same directory solr/ where webapp/ lives, and they will need to cd into webapp2/  dir. After that, they need to run npm start. That command will build the UI and then allow users to visit both Admin UI's simultaneously. This is a simple effort that will allow the community and the team to provide real-time feedback such that improvements can be made. The application will not be bundled at all in production environments in this phase. Phase one, though not entirely feature complete, is very important for our community to get familiar with the new application and its look and feel. If anyone would like to work through a first commit with me, I will talk with my company and ensure I have the appropriate time to do this work at a reasonable hour (I'm working on this projects 98% of the time, on nights and weekends). I am always available for critical bug fixes or security vulnerabilities because my company depends on Solr quite a bit

2) In phase 2, the app will start with the existing Solr start command. In this case, the admin UI would have production mode enabled, and will be served up in production mode by default. This change will result in a much smaller application, minified, that is compiled AOT, and stored in a dist  folder in the UI application's root. The application will then be packaged by Gradle into a JAR file using the gradle-node-plugin, thereby eliminating the need to have node installed on the system running the application. For implementation details on how this would look, you can view an example here: https://dzone.com/articles/integrating-java-and-npm-builds-using-gradle. You can also count on the documentation that I produce in the README to be very thorough and inclusive of steps, tactics for address errors, and alternative methods of serving the application. 

Tests

Tests will be executed in a similar fashion and as described in the link above, in the appropriate phases. By the time we reach phase 2, I hope to have worked with committers to get all the Jenkins configurations worked out to support end-to-end testing. There are some things that I need to work out before we are there, but should be done well before we are ready for Solr 9 to be merged. In fact, we should have developed a regular engagement and collaboration for working through the integrating the front end into the global CI pipeline. The added time to the UI tests should be very minimal as the application is much smaller in comparison. I will leave it to the community as to whether we make the UI tests run on every build or only for specific target. My suggestion is that if you expect the UI to work when a backend change is made, it should be added to the standard target. Admittedly, this point warrants further discussion. 

Development

As for users seeking to work on the UI, the steps will be most similar phase 1, where you need to cd from lucene-solr/solr/  into webapp2/  and run npm start. The initial build will take some time, but all subsequent changes will be reflected in the UI immediately after saving a file. If it is not a big file, it should transpile in under one second. The files are mostly structured as TypeScript files that are converted to JS at compilation, SCSS files which compile to CSS files at compilation but support standard CSS, and HTML files. There are also two files for managing packages package.json  and package-lock.json . There are thousands of node modules that are downloaded in development mode but those comprise generated code and should not be added to the repository. I have added a .gitignore to filter those and other generated files out.

Security Considerations

The current UI contains EOL JS code and libraries. They pose a variety of risks and leave Solr more vulnerable at that vector. Last year we saw an instance where using the unsupported framework led to very dangerous code living in the Admin UI application. This new framework addresses many of the web application vulnerabilities listed by OWASP. It treats all input as untrusted by default, and it has tools built in for preventing XSRF and XSSI. There are more features as well.

Eventually, it would be great to introduce some more integrated security features for the user community of Solr. Many Solr managers around the world have other teams that would like to use the Solr Admin UI, but they cannot risk those teams having access to options like deleting a collection. With this application architecture, it is very simple to expose certain modules or components based on privilege. I also have some knowledge of the authentication frameworks because I have worked with it previously and have contributed to the basic auth plugin in Solr today (with great guidance from Jan and Shalin). 

Compatibility, Deprecation, and Migration Plan

• Users will be happier.
• Both UI will exist in parallel for some time until we agree it is time to deprecate and then remove old UI.
• When we reach feature parity and the Application has been integrated into Gradle and Jenkins (Phase 2 above), it will be safe to remove the old UI and move forward.

Test Plan

The current Admin UI has no automated testing capabilities while the new Admin UI will have basic unit testing capabilities at the component level. Evnetually, in phase 2, it will support e2e integration tests.

Rejected Alternatives

Continuing with EOL AngularJS