Content

The content of the update center (nbm files, etc.) is distributed through the Apache Mirror System

These nbm files (NetBeans modules) are currently changed on each major Apache NetBeans release (or release candidate, etc.)

See Unable to render Jira issues macro, execution error. for more details about the requirements for the Update Center.

Analytics

The Apache NetBeans (incubating) project is interested in keeping track of how many modules are downloaded, so we can minimize the bandwidth consumed on each release, and understand which modules are most downloaded.

Since it's not currently possible to look at the log files of the Apache Mirror System, we've devised the following mechanism to keep track of this information:

• The Apache NetBeans IDE downloads the update center contents from https://netbeans.apache.org/nb/updates/9.0/
• This .htaccess file then redirects this request to the Apache NetBeans VM Box.
• The requested file is logged into the Apache NetBeans VM Box web server log.
• A second redirection (using the file at /var/www/html/uc/9.0/.htaccess in netbeans-vm.apache.org) sends the request to the closer.lua Apache Mirror Redirection Script that returns the URL of the closest Apache Mirror for the binary.

NetBeans AU stats counting

This is how-to setup local infrastructure for counting NetBeans AU numbers from update center webserver logfiles.

1. Prerequisites:
   
   1. mysql/mariadb
   2. PHP cli <= 5.6
   3. local web server
   4. ssh access to netbeans-vm.apache.org (to get logs)
2. Create dedicated local database.
3. Bootstrap database with schema and basic data from jchalupa.sql.gz
4. Note the DB will grow like hell, one day import brings in like 50k of records in some tables.
5. Download httpd logs for particular month from netbeans-vm:
   $ rsync -avz username@netbeans-vm.apache.org:/var/log/apache2/netbeans-vm.apache.org_access.log_yyyymm* ./
6. Put logs to location from where it can be served by local web server, keep them under /yyyy_mm/ folders for each month
7. Get the import & stats script sources from github apache/netbeans-tools/pp3/au
8. Scripts are written in super old PHP, run in cli and are compatible only with PHP up to 5.6. So you have to have php6.5 cli installed. Sorry about that.
9. Configure sctipts:
   
   1. au/config.php - for importing data; db connection, paths to logs are set there
   2.  au/db_connect.php.inc - for building stats; db connection
10. Make au/last-date-dlc file writable for user (if necessary)
11. Make sure au/last-date-dlc contains the date of last logfile you have before current import
12. Note that to have correct monthly data, you have to have last 3 months already imported. Script looks 3month back to recognize AU user. So if you count for 2020-11 month, you have to import 2020-08, 09, 10 before that.
13. Import one month of new data: in shell run
    $ for d in {1..31}; do php5.6 run-au-import.php -s dlc;done;
14. If there is new release in the imported data, you have to register it in DB
    1) get ID of new catalog: SELECT is FROM `catalogs` WHERE `path` LIKE '%XX.Y%'; replace XX.Y with new version
    2) register it: INSERT INTO `releases` (`version`, `catalog_id`, `lang`, `stable`, `delay`, `product`) VALUES ('NB XX.Y', 'ID_FROM_STEP_ABOVE^^', 'en', 'Y', 'N', 'netbeans');
15. Generate stats: in shell run
    $ php5.6 build-dashboard_v2.php -p netbeans -m 2020-09