Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migration of unmigrated content due to installation of a new plugin

Table of Contents

This page was created in April of 2017 to help modernize our system records.

SysAdmins

Goals

KAM: Apache SpamAssassin is a framework for writing rules. I deliver rules to prove the code works but I don't view that the project has to provide rules. I use this as a guidance in where I spend my focus. Beyond that, my goal with belonging to the SysAdmin group is to ensure the project is supported with modern, secure hardware and software with a bus factor greater than one.

...

Read-Write: https://svn.apache.org/repos/asf/spamassassin

Repo

Contents

Notes

sysadmins

Server and application configs

Encrypt passwords and sensitive information – NEED TO SPECIFY HOW WE WANT TO DO THIS

dns

Configs and records related to spamasssassin.org

Hosted by PowerDNS on sa-vm1.apache.org as hidden master

site

http://spamassassin.apache.org site contents

 


Bugzilla

SpamAssassin Bugzilla

...

  1. Create an account at https://wiki.apache.org/spamassassin using your full name (i.e. Jane Doe).
  2. Email sysadmins@spamassassin.apache.org to request access to the wiki:
    *Contributor only
    *Contributor and Admin

NOTE: Write access to the wiki is to anyone who has created a login name on the wiki whose name has been added to the page https://wiki.apache.org/spamassassin/ContributorsGroup

...

  1. Open an SSH tunnel: ssh -f sa-vm1vm.apache.org -L 8090:localhost:8090 -N
  2. Open web interface: http://localhost:8090
  3. Login with admin. (Password is encrypted in sysadmins/accounts.)

Zone

Server

Contact

Notes

spamassassin.org

ns2.pccc.com

Kevin McGrail kevin.mcgrail@mcgrail.com, kmcgrail@apache.org

Instant updates via NOTIFY

 


ns2.ena.com

Dave Jones djones@ena.com, davej@apache.org

Instant updates via NOTIFY

 


dns-master.sonic.net

Grant Keller grant.keller@sonic.com

Hidden slave, 5 to 10 min delay of public slaves after NOTIFY

 


ns.hyperreal.org

Brian Behlendorf

Currently not used since DJBDNS doesn't support NOTIFY or EDNS over TCP

Standards

*Apache Infrastructure standard is Ubuntu 16.04 LTS
*Cron entries should be in new standard locations /etc/cron.d, /etc/cron.daily, etc. and avoid using user's crontab
*Custom scripts should reside in /usr/local/bin if they are not direcly related to SpamAssassin processing that should be in /usr/local/spamassassin
*Symlink scripts from /usr/local/bin to /etc/cron.d, /etc/cron.daily, or /etc/cron.weekly. This provides easy discovery and future management by others on the sysadmins team.
*Scripts and cron entries should mail output to the sysadmins mailing list

...

*minotaur.apache.org - handled various build and devel related tasks
*hyperion.apache.org - likely a Solaris box that had backup data of next server
*spamassassin.zones.apache.org - DIED - was replaced with spamassassin-vm
*spamassassin.zones2.apache.org - deprecated by Infra, replaced by sa-vm1.apache.org
*spamassassin-vm.apache.org - deprecated by Infra, replaced by sa-vm1.apache.org
*buildbot, ruleqa, etc. are aliases of above deprecated servers

Servers

Hostname

Function

Software

Configs/Location

Resource/URL

SVN Location

incoming

apachesf.

apache

sonic.

org

net

Donated by Sonic

 

 

 

 

CentOS 7


apachesf.spamassassin.org (64.142.56.146)


colo.sonic.net

Retired



76.191.162.2


trap-proc.spamassassin.org

Retired



a.k.a spam-trap.spamassassin.org (192.87.106.247)


sa-vm1.apache.org

DNS Hidden Master

PowerDNS

/

sa-vm1.apache.org

DNS Hidden Master

PowerDNS

/

etc/powerdns/pdns.d/pdns.local.conf

spamassassin.org

dns (webserver API key redacted)

 


Rsync Mirrors

rsyncd

/etc/rsyncd.conf

rsync.spamassassin.org

trunk/build/automc/etc-rsyncd.conf

 


Web Server

apache2

/etc/apache2/sites-available/automc.conf

updates.spamassassin.org

trunk/build/automc/automc-apache2.conf

 

 



apache2

/etc/apache2/sites-available/automc.conf

ruleqa.spamassassin.org

trunk/build/automc/ruleqa.cgi

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="

4763f79f

1d53d356-

a4d3

108f-

493b

4add-

949c

b325-

edcf887c742a

d37f2551dcd3"><ac:plain-text-body><![CDATA[

 


SaUpdateMirrorSetup

svn

rsyncd [updates] for mirrors

spamassassin.apache.org/updates

[site/updates

http://svn.apache.org/repos/asf/spamassassin/site/updates/]/MIRRORED.BY

]]></ac:plain-text-body></ac:structured-macro>

 


Nightly Masscheck

cron/scripts

/usr/local/spamassassin/automc/rsync/tagged_builds

ruleqa.spamassassin.org

trunk/backend/nitemc/README

 


RuleQA web UI

cron/scripts

/usr/local/spamassassin/automc/html

ruleqa.spamassassin.org

RuleQaApp

Backups

An old backup exists in sa-vm1.apache.org:/usr/local/spamassassin/backups/spamassassin-vm. It's a large bzip'd tar file so make sure you don't extract it and fill up the filesystem.

...

Wiki Markup
 *25 2 \* \* \* automc *~/svn/trunk/build/mkupdates/do-stable-update-with-scores*
  **~/svn/masses/rule-update-score-gen/do-nightly-rescore-example.sh*
   **~/svn/masses/rule-update-score-gen/generate-new-scores.sh*
    *uses ~/tmp/generate-new-scores for SVN work area
    *sorts out the usable corpus from the latest 'SVN revision' at the top of the submitter's log file which should match the latest tagged build of SVN rules
    *checks $\{REVISION\} LINE 123 NEEDS IMPROVEMENT!!!  THIS SVN REVISION NEEDS TO BE CLOSELY TIED TO THE REVISION THAT WAS STAGED IN THE MASSCHECK RSYNC DIR.
    *checks the sorted corpus for a minimum number of valid contributors and ham/spam
  **~/svn/trunk/build/mkupdates/mkupdate-with-scores*
   *uses ~/tmp/sa-mkupdate for SVN working area
   *gets latest SVN $\{REVISION\} from rulesrc/scores/score-set\*
   *masses \-> perl Makefile.PL && make (complete build of SA and test)
    *perl hit-frequencies
    *garescorer - compiles and runs it, requires build/pga
  *sends email if not enough masscheck submitters or usuable ham/spam for the latest SVN revision
  *creates $\{REVISION\}.tar.gz $\{REVISION\}.tar.gz.sha1 and $\{REVISION\}.tar.gz.asc in /var/www/automc.spamassassin.org/updates for mirrors to pull
  *updates DNS TXT entries \[0-3\].3.3.updates.spamassassin.org and 0.4.3.updates.spamassassin.org -- versions >= 3.4.1 have a CNAME to 3.3.3.updates.spamassassin.org
  *Script rewrite notes:
   *Make each primary step modular since these steps are commmon in other scripts
   *Should check for minimum contributors of ham/spam up front and not waste resources if requirements not met
   *These 3 scripts above all share the same temp working dir.  This should be determined from config file or relative path of user's home dir for flexibility.
   *Should be able to run the ham/spam processing in parallel and merge the results together to cut this time in half
   *Temp working dir for the corpus should be persistent so the rsync copy will be faster.
   *Usuable corpus symlink setup could be improved.  Invalid stale corpus should be removed into an archive/excluded dir.

...

Wiki Markup
 *30 8 \* \* \* automc *~/svn/trunk/build/mkupdates/run_nightly* > /var/www/automc.spamassassin.org/mkupdates/mkupdates.txt
  *Currently $\{SA_VERSION\} = "3.4.2"
  *$\{REVISION\} = latest SVN revision  THIS NEEDS TO BE ADDRESSED!!!  NEED TO PREVENT REVISION FROM MESSING UP THE MASSCHECK PROCESSING.
  *creates new rules/active.list
  *commits new rules/active.list
  *runs spamassassin lint against the updated rules and checks in a tagged version of 'sa-update_$\{SA_VERSION\}_$\{TSTAMP\}'
  *commits "promotions validated" and emails dev@spamassassin.apache.org
  *creates $\{REVISION\}.tar.gz $\{REVISION\}.tar.gz.sha1 and $\{REVISION\}.tar.gz.asc in /var/www/automc.spamassassin.org/updates for mirrors to pull
  *updates DNS TXT entries \[0-3\].3.3.updates.spamassassin.org and 0.4.3.updates.spamassassin.org -- versions >= 3.4.1 have a CNAME to 3.3.3.updates.spamassassin.org
  *Script rewrite notes:
   *Uses many of the same primary steps previous section so reuse the code and not have to maintain multiple versions
   *Should be turned into generic script that can be run on demand via SVN trigger/polling

...