Page History

Initial notes from trying to migrate Ref Guide from Jekyll to Antora Antora website: https://antora.org/

Docs: https://docs.antora.org/antora/23.30/

Cassandra's branchBranch:  https://github.com/ctargettapache/solr/tree/antora-migrationjira/solr/solr-ref15556-guideantora/ (does not yet remove all vestiges of Jekyll)

Antora Go-No Go Decision Points

There are 2 main areas of concern that I would want everyone to be aware of before deciding to go forward with Antora as a replacement for Jekyll:

1 - Differences from our usual workflow

There are many attractive things about Antora for us, but a few things that are really different from our usual workflow:

1. Because it's Git-based, once we have it all set up it will by default only build from the files checked into the origin repo in Git. This makes it a little bit harder to build locally, although there is a local "author" mode available. 
2. I'm not clear if Jenkins (ci-builds) will be able to build this. edit 24 Nov: I set up a Jenkins job to test this out and it works seamlessly: https://ci-builds.apache.org/job/Solr/job/solr-reference-guide-antora/. It's currently run on demand only, and pushes the output to the nightlies.apache.org server.
3. Validation that Gradle plugin is viable (using this may just solve the Jenkins question)

2 - UI Bundle requirements

The design of the UI (see below) assumes you will keep your UI files in a repo separate from your docs (since it supports multiple Git repos), and you "release" a .zip of your built UI files bundle which is consumed by the build.

1. If we don't want to put the UI into a separate repo, we could publish to nightlies.apache.org, like we do with the Lucene snapshot.
   1. edit 24 Nov: To test this out I built the current UI from the branch locally and pushed it up to nightlies.apache.org, at https://nightlies.apache.org/solr/draft-guides/ui-bundle.zip. The playbook.yml file in the branch has been updated to pull from this location. To me this feels like the right way to do this, but it should probably have a Jenkins job and be versioned once the dust settles.
2. We could also commit the .zip to the solr  repo, but that also feels like something we don't really like to do (or I could be wrong?). And it could drift by branch....
3. Or we could have a separate repo for the UI files that has "releases" stored in Github? This has felt rather heavy to me.

Remaining major TODOs

This list will probably grow, but this is what I know is still left to do:

• Finish the UI updates
  • Styling, header, footer
  • DRAFT watermark and styling
  • Add STEM support (do-able, just not done)
• Write an Antora extension to do validation like orphan checking (maybe also other things? could be added later)
• Figure out how we will populate variables like dependency versions we get from versions.lock  today, and other things today we automatically populate like the version in the header, etc. I wonder if Antora's new extensions could do this?
• Fix how all inter-page links are constructed (from << >>  to xref: ) to be sure inter-module links are correct - DONE 12/2021
• Remove pages that are primarily navigational which today make our hierarchy look duplicated. - DONE 12/2021
• Since 9.0 will have new page names and this change will create new URL paths, we probably really should do something about redirecting old pages

Initial Notes from Early Antora Investigation

(Jun-Aug 2021)

Antora Design Goals

• Designed FOR multi-version technical documentation
• Git-based

Gradle Integration

Antora is a Ruby gem which requires Node.

...

• Basic functionality, not sure if more is needed?
• Downloads Node and Antora and all dependencies
• This creates a node_modules directory and a package-lock.json file - do these need to be committed, or aded to .gitignore? (On my branch I committed package-lock.json  since I read docs that said we should)
• There is an open issue for Antora to make it "official" (which states the plugin still needs hardening): https://gitlab.com/antora/antora/-/issues/632

As of 24 Sep 2021, the 3rd-party Gradle plugin has been accepted and integrated as an official Antora project, although it's in early days and has not yet had a release: https://gitlab.com/antora/antora-gradle-plugin/

Antora Concepts

Helpful: https://matthewsetter.com/antora/three-core-concepts/ (but out of date in terms of requirements)

Playbook

A playbook is a YAML file (could also be JSON or CSON), usually playbook.yml, which defines global attributes of the documentation site. Examples: site title, URL, starting page, sources of content (Git repo & branches), overall UI theme, output directories, Asciidoc attribute settings (attributes can also be per-module), etc.

...

The playbook can reference repos that are local or in GitHub or GitLab, etc. Specific branches can be built or use HEAD for whatever branch is checked out at that time.

Component

All projects have at least one component, defined by an antora.yml file stored at the "content root". Its presence indicates to Antora that it should find content files in this location (organized under a modules  directory). The antora.yml file must include a name, but can also include a version.

...

For the Ref Guide, we really only have 1 application (Solr) so this isn't an intuitive concept for us. However, if we published several different Guides for different applications (perhaps like "Core Solr", "Solr Operator", etc.) they could each be different components and have different versions, content locations, etc.

When Antora builds the documentation, all configured versions are built at the same time. The latest version can be called "latest" (or "current" or whatever we want) allowing URLs to be created with that in the path.

Module

A module is an organizational unit within a component. The primary top-level sections of the new Ref Guide could be considered modules.

...

A special module named ROOT module is optional, but used for pages which have no formal module - like primary navigation pages, index.html, etc. Pages placed under ROOT do not have any module name in the path.

This module construct means that we will need to change our page-to-page links from a structure like <<pageName.adoc#anchor,Title>> to xref:module:pageName.adoc#anchor[Title]. One plus here is that Antora will automatically populate the page title if it's not defined in the xref, and in 3.0 the build has validation to make sure page references can be resolved (we had to validate this manually with Jekyll).

There is not yet support for orphan checking (all pages are referenced in a nav), but the Antora community is working on support for extensions and one of the first use cases they want to document as an example is this sort of validation, so it's likely there will be some way to do this within Antora soon.

Family Directories

Asciidoc files are always stored in the pages  family directory. Images should be in the images  family directory.

One interesting thing about the family directory structure is that paths do not need to be included when referencing an item in another family directory. For example, to include an image that's in the images  family, one would only need to reference the module & filename, as in image::module:filename.png[], and not worry about determining the full path.

Navigation Files

Navigation files can be located anywhere (defined in the playbook), but a custom is to make a nav file for each module (how could we automate this? Maybe we don't...how often do we really move stuff? If we did manage it manually, we could probably get rid of a lot of nav-only pages.).

If we did decide to now manage this manually, we could likely remove many pages which are navigational only.

Examples

Because Antora is able to simply include files by name only (and doesn't require relative or absolute paths), it has a limitation where any files outside the defined family directory structure can not be included at all.

This poses a problem for the Ref Guide, as we have several code examples (particularly for SolrJ, but also others) that pull from real tests which are not located in the Ref Guide directory tree.

However, with Antora 3.0 symbolic links will be supported, so this provides a workaround. In the branch linked above I have tested this out and it works well, although we should test various dev systems to be sure the ones created on my OS work in other OS's (particularly Windows). Since this is a feature of Antora for v3, they are also testing this extensively.

Build Options

By default build output is in antora_home, which is the location of the playbook.yml file.

...

Recursively cleaning the build area is an option that can be added to the playbook so it doesn't need to be defined at Gradle runtime.

UI Bundle

Today with Jekyll we create each version of the Ref Guide with all pages, and in the directory structure that's created, each version has its own CSS directory, font directory, javascript directory, etc.

...

As of version 3 of Antora (TBD, in the works), we will be able to refer to an expanded directory of UI files instead of a .zip.

UI Design Changes

Some notes

The default UI of Antora does not include Bootstrap, which is probably a good thing. The main thing we use Bootstrap for today is the tabs layout on pages such as https://solr.apache.org/guide/configsets-api.html#configsets-list. There is an Antora extension to do the same thing: https://gitlab.com/antora/antora-asciidoctor-extensions/-/tree/master/tabs-block, which may be a viable alternative. Would need to add it as an extension in our playbook. Couchbase docs have an example of syntax & output: https://docs.couchbase.com/home/contribute/tabs.html

Misc Notes

• Source highlighting only available through highlight.js
• STEM support needs to be added to the UI (it's not OOTB) because it relies on MathJax

...