THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
- The Apache NetBeans IDE downloads the update center contents from https://netbeans.apache.org/nb/updates/9.0/
- This .htaccess file then redirects this request to the Apache NetBeans VM Box.
- The requested file is logged into the Apache NetBeans VM Box web server log.
- A second redirection (using the file at
/var/www/html/uc/9.0/.htaccess
in netbeans-vm.apache.org) sends the request to the closer.lua Apache Mirror Redirection Script that returns the URL of the closest Apache Mirror for the binary.
NetBeans AU stats counting
This is how-to setup local infrastructure for counting NetBeans AU numbers from update center webserver logfiles.
- Prerequisites:
- mysql/mariadb
- PHP cli <= 5.6
- local web server
- ssh access to netbeans-vm.apache.org (to get logs)
- Create dedicated local database.
- Bootstrap database with schema and basic data from jchalupa.sql.gz
- Note the DB will grow like hell, one day import brings in like 50k of records in some tables.
- Download httpd logs for particular month from netbeans-vm:
$ rsync -avz username@netbeans-vm.apache.org:/var/log/apache2/netbeans-vm.apache.org_access.log_yyyymm* ./ - Put logs to location from where it can be served by local web server, keep them under /yyyy_mm/ folders for each month
- Get the import & stats script sources from github apache/netbeans-tools/pp3/au
- Scripts are written in super old PHP, run in cli and are compatible only with PHP up to 5.6. So you have to have php6.5 cli installed. Sorry about that.
- Configure sctipts:
- au/config.php - for importing data; db connection, paths to logs are set there
- au/db_connect.php.inc - for building stats; db connection
- Make au/last-date-dlc file writable for user (if necessary)
- Make sure au/last-date-dlc contains the date of last logfile you have before current import
- Note that to have correct monthly data, you have to have last 3 months already imported. Script looks 3month back to recognize AU user. So if you count for 2020-11 month, you have to import 2020-08, 09, 10 before that.
- Import one month of new data: in shell run
$ for d in {1..31}; do php5.6 run-au-import.php -s dlc;done; - If there is new release in the imported data, you have to register it in DB
1) get ID of new catalog: SELECT is FROM `catalogs` WHERE `path` LIKE '%XX.Y%'; replace XX.Y with new version
2) register it: INSERT INTO `releases` (`version`, `catalog_id`, `lang`, `stable`, `delay`, `product`) VALUES ('NB XX.Y', 'ID_FROM_STEP_ABOVE^^', 'en', 'Y', 'N', 'netbeans'); - Generate stats: in shell run
$ php5.6 build-dashboard_v2.php -p netbeans -m 2020-09