Ref Guide Re-Org 2021

Planned release: Solr 9.0

Jira issue:  Unable to render Jira issues macro, execution error.

Overview of Major Changes

1. Extensive reorganization of nearly all pages.
2. Extensive page re-naming and content consolidation (leading to several page deletes).
3. Split the Solr Tutorial into parts.
4. Standardization of "standalone", "leader/follower", "legacy".
5. New formatting and layout styles.

New Section Organization

The primary goal in thinking through a better way to organize the content was to reduce the number of "top-level" categories to a significantly smaller list, but this wasn't just an exercise in moving sections. There are changes to the 2nd, 3rd, and 4th level categories. 

I decided to present each top-level category as a "Guide" in itself. Of course there are inter-connections and things that just won't fit cleanly into a single place, but ultimately I tried to conceive of each guide as able to stand to a degree on its own.

There are now 5 main top-level categories (screenshot below):

• Getting Started
• Deployment Guide
• Configuration Guide
• Schema and Indexing Guide
• Query Guide

I've put the Solr Upgrade Notes (and its children pages "Major Changes in ...") on the top-level, so it's technically a 6th category. This could go into Deployment, but I thought it was important to keep it at the top since it's a common thing people need. I'm willing to discuss alternatives, though.

The Admin UI section has had a dramatic overhaul. Instead of a section dedicated just to the UI, I've moved all of the child pages to fit in with their respective topics. So, the page on the Logging screen in the UI has been removed and the content moved to the page "Configuring Logging" (which itself has a new name, more on that to follow). There is still a page dedicated to the UI, but it covers general "here's the UI" concepts, and links to the pages or sections that cover each screen.

The Query Guide is split into three sub-sections: Query Syntax & Parsing; Enhancing Queries; Controlling Results. I'm not positive this conceptually works, but the idea was that it needs some sub-organization and those seemed to fit with what that section generally does. Again, other ideas are welcome.

Page Renames & Deleted Pages

I've renamed dozens of pages to make them simpler, shorter, and more straightforward. The renames include the filename as well as the name that's displayed to users.

I generally renamed pages that started with "The", "Solr", and gerunds. A few examples:

• "The DisMax Query Parser" becomes "DisMax Query Parser"
• "Solr Configuration Files" becomes "Configuration Files"
• "Detecting Languages during Indexing" becomes "Language Detection"
• "Making and Restoring Backups" becomes "Backup and Restore"

etc. I kept some "-ing" words in titles though, and also some "Solr", etc. (e.g., "Scaling Solr"). It isn't my goal to ban that usage, just that in a lot of cases the page topic is equally or more obvious without them.

I did, however, rename almost all of the pages under the current "Configuring Solrconfig.xml" which were named after the solrconfig elements but are now named after the thing you configure in those elements. So, "Query Settings in solrconfig" becomes "Caches and Query Warming", etc.

Lists of all renames and deletes so far will be attached to the Jira issue.

Solr Tutorial Split

The massive Solr Tutorial has been split into 4 pages - an intro and then one for each exercise - and there is a new dedicated section under "Getting Started" for Tutorial content. The AWS Tutorial and the "Getting Started with SolrCloud" page have been moved here (the latter being renamed to "SolrCloud Tutorial"). Screenshots below showing subsection list.

I did not try to improve any of the tutorial content, so if possible let's not discuss now about whether the exercises are appropriate or easy or whatever problems you have with them. It's not really relevant to this overall exercise IMO.

I decided I was only prepared to go so far as to create a space for tutorial content, and hopefully someone will take the hints I'm dropping by having split the massive tutorial into parts (which are that we should explore several smaller tutorials and maybe thinking about it that way will be easy enough for us to tackle going forward). Approaching it in smaller bites might make it easier to make some progress on this gap.

Unless you plan to fix the whole situation by Solr 9, I would rather questions about the tutorial in general not hold up the hundreds of other changes I'd like to make by merging what I've got so far.

Standardize Cluster Nomenclature

I've standardized our naming around this, which we need to discuss.

The SolrCloud/Legacy problem was a big reason I wanted to embark on this project, so I spent hours thinking about this. I'm still thinking about how best to organize this content.

The first thing I realized is that we use the terms "SolrCloud" and "Standalone" as though they are the only two distinct modes, but "standalone" really means one of two different types: a truly single-node installation and a cluster (usually single-sharded) that has multiple replicas. So we talk about this as though there are only 2 modes of running Solr, but from a documentation standpoint I think we need to be clear that there are really 3.

The second thing I realized is that SolrCloud and "leader-follower" share several similarities. Both have shards. Both have replicas. Both have leaders and cores. The differences are vast, of course, but they both share some fundamental principles, and in a few cases even sharing code.

So I initially thought I would actually mix the content together - a section of content that had pages about replicas would have a page about SolrCloud replication and another about "legacy" replication, etc. But then as I did that it really didn't work - SolrCloud is much more complex because of what it offers, and with improved naming it makes sense to keep them organizationally separate.

To bridge the gap I've written a new page "Solr Clusters" which attempts to differentiate these in clear terms. In that document I introduce the concept of a "user-managed" cluster (based on discussion we had last year, I just settled on it), and discuss the primary concept behind each straightforwardly and in a comparative way (with sentences like "Unlike in SolrCloud, a user-managed cluster does not use ZooKeeper", etc.). This led to these changes:

* "Legacy", "Leader/Follower", and "Standalone" when used to describe a cluster of Solr nodes that isn't SolrCloud becomes "User-managed cluster"
* "Standalone" where it refers to a one node installation becomes "Single-node installation"
* "SolrCloud" stays "SolrCloud" but I changed several instances of "SolrCloud mode" to "SolrCloud cluster"

This allowed me to review all the places in the Ref Guide where we talk about "standalone mode"/"legacy"/"leader-follower" and make it clear what exactly we're talking about. I've introduced sentences like "In a user-managed cluster or a single-node installation..." in several contexts where previously we only talked about SolrCloud or referred to it as "legacy" or "standalone".

I did it this way so we could discuss if anyone has objections to thinking of it this way as a project. And if so and we agree on some other phrasing, it will be easy to change.

As an opinionated side note - I really hesitate to initiate a conversation about this. Every time it comes up we throw words up against the wall like a brainstorming session but we've "brainstormed" this for years with no forward movement and users are none the better for it. We need to just pick something and move forward with it, and that might mean some people just go with someone else's suggestion and not try to put up their own ideas.

Note I did not tackle renaming SolrCloud - that terminology is everywhere in the code (classnames, etc.), so until we do a project-wide change I think the docs should keep using it.

New Formatting Elements

There are several new file and display formatting elements I've introduced:

• New section layouts in "parent" pages (the ones that are mostly "here is what's in this section"). Where there are subsections of subsections the subsection list lives only on one page - the lowest level - and is tagged so it can be automatically included in all parent pages. This allows us to have descriptions consistent but maintained in only one place.
  • For example, the "Query Guide" has a subsection "Query Syntax and Parsing" which has several child pages. The "TOC" (as it were) is maintained on "Query Syntax and Parsing" and pulled into "Query Guide" only at build time.
  • This is also now in a table, and I've defined specific formatting for this purpose to display it in two columns instead of our previous long lists.
  • Multiple child page subsections have h3-level headers to differentiate them.
  • A couple of examples in screenshots below.
• All sentences start on new lines. I've missed some; IMO it's not a critical component, but can be continued ongoing.
• New format for parameters (only very partially done). See screenshots below.
• The icons in the left navbar are changed from arrows to + and - symbols.
• External links all now really open in new tabs (as has long been indicated by the icon that follows the URLs) (uses JS).
• Guide home page (index.adoc) redesigned a bit to take up less real estate (screenshot below). Download button has moved to top navbar as a link.
• Simplified the link text in top navbar "Resources" section; added "Contribute" to link to ref guide contribute docs in solr/dev-docs (TBD with SOLR-15458).

Screenshots