Table of Contents

Static Site Generation/Build Tools

Reasoning

Why would we want a site generation/build tool versus statically maintaining a site? Branding and time. Imagine we had to edit X number of files to replace a footer or header versus editing a single file. What if we had to edit X pages just to replace the site logo? This would be horribly time consuming, and as we grow here at Apache, will become great overhead impacting accuracy and possibly releases.

Other Apache projects use build tools. The Groovy project built their own some time ago. There are static site tool projects which use other Apache projects such as Groovy and FreeMarker with JBake being such an example, and hopefully one of these will work for our needs, to promote the Apache tools, and can be the first attempt at a successful static site generation project for NetBeans.

There are also other tools which use Ruby, Python, or Node JS. Various tools have been reviewed, and do quite similar things. If a Java tool such as JBake which uses or allows the use of different Apache projects can not be used, then perhaps one of those can be used, and in that case we should consider a Node JS build tool as the browser front end market and developers have been using Node JS stacks more frequently, and perhaps would be closer to the problem in this case, and NetBeans has great support for Node JS projects.

Possible and Reviewed Static Site Generators

As mentioned in the reasoning, there were various reviewed tools built on different stacks such as Java, Ruby, Node JS, and Python. In the process of this work, rolling our own generator was not considered, but could certainly be considered and reviewed, as the Groovy project did this as an example. All reviewed tools market themselves as Static Site Generators, and was the main criterion used to determine if a project should be reviewed, along with having an Apache approved OSS license.

Criterion included (weighted by order)

1. Does the project consider itself a static site generator or advertise it could, versus mainly just blogs, and without rolling our own
2. Does it support transferring the current site content with less effort; not too much manual changing, or could be fairly trivial transformed using scripting such as Groovy and JSoup to convert current HTML into content HTML less Markdown, AsciiDoc, or other
3. Does it support HTML content (necessarily different than templates); goes with 2 above
4. Does it support common web and document formats, standards, and idioms
5. If the tool had deficits in the above, how easy would it be to integrate with common Java build tooling to fill the gaps 
6. Simplicity of use; perceived
7. Would the tool be familiar to NB project maintainers (the Java ecosystem, or at least close enough if not enough tools could be found such as tooling supported by the IDE)
8. What template engines it supported out of the box or with supported plugins (those in the Apache ecosphere were rated higher), and ease of use

There are certainly others, but the following is a good sampling of some of the top ones from their given categories.

JBake (Java based)

http://jbake.org

Java seems to have a quite limited SSG market, and thus far, without rolling ones own with Groovy, JBake seems the best. It also uses various Apache projects or ones licensed with the Apache license. It may not necessarily be the best SSG out there, as Jekyll, Metalsmith, and Nikola have a lot of features, and seem quite robust. However, it does accept html content input, allows Java developers to use Java ecosystem templates such as Thymeleaf, Groovy, and Freemarker, and seems to integrate nicely with Gradle which allows some of its limitations to be easily overcome by adding some extra preprocessing by way of other Gradle plugins and tasks such as SASS processing.

HTML as content input out of the box is fairly important as it hopefully allows those working on site transition from netbeans.org to a static version to do less rewriting from .html files to .md (or other). This was listed in the criteria.

JBake also seems quite simple to work with, and is fairly trivial to get setup with a basic site. One aspect which is missing, and could be subjective, is the ability to have JavaScript dependencies annotated and downloaded from the tooling. That said, this can be combined with a Node JS sub-project which could be driven with Gradle and a plugin. This same method could be used for minification and other JS tooling. This similar to the SASS plugin mentioned before.

Jekyll (Ruby based)

https://jekyllrb.com

Jekyll is a highly popular SSG. It is well documented, and supports quite a lot of features out of the box. It does have some specific structure specifics enforced on the project and content however. It also seems quite a large project with much documentation which can be a catch 22 for this problem at the start, but may be something to look at once we have some initial SSG in place. Jekyll doesn't seem to support HTML files as content or if it does, it is a plugin which was not easily located, and expects a Markdown file.

Metalsmith (Node JS)

http://www.metalsmith.io

Metalsmith is a quite simple Node JS which like gulp uses a streamed build approach. There are various plugins one can include in their build file which them supports various formats. The HTML input for content was not yet found, but may exist. This seemed to expose an issue with the documentation not being organized well. The project itself seems to support quite a lot of functionality including layouts and tools which front end developers find useful. This would be another tool to look at once we have something in place. It seems it may take more time to get up to speed on it, but Metalsmith looks like it could be promising. 

Wintersmith (Node JS)

http://wintersmith.io

Wintersmith seems like it could be a good tool, but its documentation is quite lacking, or possibly just organized badly. It does seem to have a hearty plugin system to support a lot of what one would want to do, but I did not find quickly that HTML content can be used easily outside of templates/layout processing. It could be worth another look in the future, but Metalsmith seems further along.

Hyde (Python)

http://hyde.github.io

Hyde is an interesting project in that it is said to be Jekyll's evil twin. That said, the documentation was quite lacking, contrasted to the great docs of Jekyll. Too, if it is the evil twin of Jekyll, then one can assume it supports quite a lot of functionality undocumented, but this could be a guess. It also uses some nonintuitive notions of "media" as it assumes CSS and JS would naturally fall under the same asset concept where as this may not be accurate at all, and thus seems to force this nonintuitive structure on projects.

Nikola (Python)

https://getnikola.com

Nikola would probably be better described as Jekyll's evil twin in the Python world versus Hyde as it is robust and well documented. It also seems to impose less limitations on structure, and is highly configurable. It also supports HTML content input like JBake out of the box which is important per the current criteria. Nikola is certainly an option for the future.

Current SSG Suggestion

The current suggestion is JBake along with Gradle to fill in any gaps. Gradle can initially provide SASS preprocessing capabilities along with any others as mentioned in the section on JBake above. JBake also supports less changes to the current NB site structure while allowing us to rebrand the site as it allows a flexible structure and HTML content. We will need to write some scripts to perform some content preprocessing as JBake supports HTML fragments in .html files for content, but this should be fairly trivial with Groovy and JSoup, and would be necessary regardless of the chosen SSG. Patterns are being identified which can be baked into such scripts. Then, JBake can be used to merge the new site look and feel with the old site structure as much as possible.

It is suggested future articles, posts, and content, beyond the initial migration, use Markdown as the editing language choice as it is certainly easier to type and edit, and the templates and engine will handle the content layout along with transforming Markdown to HTML for the site fitting the templates. This gives a good starting or conversion point plus a better editing experience for the future.

Possible Future SSG Suggestions

Introduction

This document describes how to work the current NetBeans website(s) http://www.netbeans.org and http://platform.netbeans.org (and others possibly added later) into the new site being developed for the move to Apache. The new site has a specific look and feel which we would like to keep uniform. One of the rules at Apache is the site can not have a "back end server" other than 3rd party services such as discuss and mailing lists etc as this imposes things in infra which they can not afford to manage at this time, so a static site generator (SSG) will be used to allow for templates, writing, and blog style editing.

This is a pattern used by other projects at the ASF. Groovy as an example wrote their own. This document describes the site layout, how to use the SSG JBake in this context, a set of Gradle build files, and set of JSoup dependent Groovy scripts used to crop or scrape document content from existing pages to be placed into content files which get embedded in the new web site. Eventually the Groovy scripts will no longer be needed, and should go away; they are essentially tools for now.

Domain Names

The Apache NetBeans Incubating project has the domains http://netbeans.org and http://netbeans.apache.org. The suggestion at the moment is to redirect netbeans.apache.org to netbeans.org. If that is frowned upon, then it is suggested both netbeans.org and netbeans.apache.org have the same content, but have the domains point to the same information in site configuration (assuming this is httpd at Apache). This allows for the Apache domain branding to be used, and it allows for the history of NetBeans to be preserved which has proliferated for ~20 years or more at this pointPer the SSGs reviewed it seems Nikola or Metalsmith would probably be good evolutions if JBake and Gradle can not keep up with where the projects needs travel, and with a little more time with those tools, they seem likely to be successful. Jekyll may also be a good option, but it would depend on its allowed structures, but it certainly has many plugins and tools which front end developers find useful, but Nikola supports HTML content out of the box, and Metalsmith may be able to support such a notion going forward or already by way of a plugin along with many tools front end developers find useful.

Envisioned Generated Site Layout

As much of the current NB site directory structure layout should be used as possible. This means things such as https://netbeans.org/kb/index.html and https://netbeans.org/kb/trails/java-se.html will continue to work if folks have permalinked them. However. There are certainly sections which may not translate directly to Apache or our current point in time, and we'll need to redirect to the home page in such cases (or something more reasonable). Too, assets and media should be rearranged. Currently CSS and JS are scattered in nonintuitive places such as under /images_www, and the CSS is under various directories. This should be better organized into either a top down layout such as:

...

Or, the site should be organized more along the lines of web components. At the moment, the current site seems a mild hodgepodge albeit those more web component type things reside under sub-structures in /images_www, and could still be considered more top down. For the sake of getting the site over to Apache, it is suggested a top down approach matching either of the above can be fine, and either will require the same amount of effort for scripted replacements of the current content to get it into the needed SSG structure.

The current site structure follows the below, less the asset changes mentioned above:

TBD

New Site Plan

Converting the existing site to JBake will require a Gradle build setup, JBake configuration, a project structure, templates and layout, choices between page types and site sections, and some scripts to modify the current "content" files to match what JBake needs. This section will layout what that is. The output of these scripts, builds, templates, and content should deliver the envisioned generated site layout. This is obviously just a starting point at this time to be tweaked with other community members, but gets the SSG in place, to then be styled and beautified as needed.

This has been started at https://github.com/wadechandler/netbeans-static-site which is a temporary repository to get the build in place along with trimming out bloat (think Git history) which will not be needed .or allowed at Apache. The build and site is documented in a README.md file, and all contributors are encouraged to read it fully as a first step.

There are also other aspects to keep in mind. There are portions of the old NetBeans site related to very old information such as Enterprise support and competitions which will not mean very much or as much at Apache. Some portions of the site, such as downloads and issue uploading, are not something we can manage from the site at Apache either. We will need to house the information and services in different places. So, these things will need to be replaced, and part of the overall plan will include this.

For issues related to the website see this this query https://issues.apache.org/jira/issues?jql=project%20%3D%20NETBEANS%20AND%20status%20in%20(Open%2C%20"In%20Progress"%2C%20Reopened)%20AND%20component%20%3D%20website

Script Conversion of Knowledge Base Sections HTML Files to Site Content

The kb or Knowledge Base section of the site needs converted. This is best handled with scripting to match what the site project and SSG need. See 

Script Conversion of Community Sections HTML Files to Site Content

The "Community" sections of the site needs converted. This is best handled with scripting to match what the site project and SSG need. See 

Script Conversion of Features Sections HTML Files to Site Content

The "Features" sections of the site needs converted. This is best handled with scripting to match what the site project and SSG need. See 

Script Conversion of Platform Sections HTML Files to Site Content

The "Platform" sections of the site needs converted. This is best handled with scripting to match what the site project and SSG need. See 

Replace Downloads Service

We need to look at other Apache sites, and see how they integrate downloads. The current NetBeans site has a special download section which allows one to selected different types of installers/products such as PHP, Web Apps, C/C++, Java, Full, etc. It needs to be decided how this can/will work, and if Apache has something similar which we can use.  See 

Replace Downloads Section

We need to look at other Apache sites download sections, and work up content and a template for this. See 

Replace Updates Section

The current NetBeans site has the releases update catalog URLs held under an "updates" folder. This doesn't really make sense to keep under a static site. This should be under some more dynamically updated location much like the plugin portal update center file should be kept as well as downloads. We need to understand what to do with this, and make it happen. For the updates links, perhaps it makes sense to have them under areas like downloads where they are accessible by mirrors for the IDE to help spread the load (or perhaps stored in the Maven repository) See 

Replace About Section

The about section needs to be updated to match the project at Apache. See 

Replace UI Gestures Service

There is a UI gestures module in the NetBeans IDE which has currently been removed. This is something really good to have. A new service needs to be built based on the information collected from the service, and then moved to Apache. This will require something being run under infra. See 

Replace UI Gestures Section

Other than the service mentioned above, the issue also references reports, graphs, and statistics shown on the NetBeans web site here http://statistics.netbeans.org/analytics/. This seems extremely useful to the community when deciding where to focus, issues, etc. See 

Replace Plugin Portal Service

See 

Replace Plugin Portal Section

See 

Relocate DTDs and NS (XML Schemas)

The NetBeans site has various DTDs and XML Schemas under the directories dtds and ns. These resources are more related to different projects in NetBeans (such as project and file support). This does not seem the proper place for these things. See 

Update or Replace Books Section

The books section is dated and really needs to be organized in some way (perhaps some way to filter on page) See 

Remove or Replace Switch Section

The NetBeans site has a section called "Switch". Do we want this at Apache? It is probably all in good fun and all, but still, it needs updated at a minimum, and too, do we want to keep it up? See 

 TBD