...

You will need to install Metron first. Today, there are three options to install Metron: Metron Installation Options. Chose any of the options and below instructions should be applicable to all three install options given the following environment variables that you will need to plugin with your own values:

• KAFKA_HOST = host where a Kafka broker is installed
• ZOOKEEPER_HOST = host where a Zookeeper server is installed
• PROBE_HOST = Host where your sensor, probes are installed. If don't have any sensors installed, pick the host where a storm supervisor is running
• SQUID_HOST = Host where you want to install SQUID. If you don't care, just install on the PROBE_HOST

How to Parse the Squid Telemetry Data Source to Metron

...

• bro
• enrichment
• pcap
• snort
• squid
• yaf

Step

...

2: Install Squid

1. ssh into $SQUID_HOST
2. Install and start Squid:

   sudo yum install squid
   sudo service squid start

3. With Squid started, look at the the different log files that get created:

   sudo su -
   cd /var/log/squid
   ls

   You see that there are three types of logs available: access.log, cache.log, and squid.out. We are interested in access.log becasuse that is the log that records the proxy usage.

4. Initially the access.log is empty. Let's generate a few entries for the log, then list the new contents of the access.log:

   squidclient http://www.cnn.com
   squidclient http://www.nba.com
   cat /var/log/squid/access.log

   In production environments you would configure your users web browsers to point to the proxy server, but for the sake of simplicity of this tutorial we will use the client that is packaged with the Squid installation. After we use the client to simulate proxy requests, the Squid log entries should look as follows:

   1461576382.642    161 127.0.0.1 TCP_MISS/200 103701 GET http://www.cnn.com/ - DIRECT/199.27.79.73 text/html
   1461576442.228    159 127.0.0.1 TCP_MISS/200 137183 GET http://www.nba.com/ - DIRECT/66.210.41.9 text/html

5. Using the Squid log entries, we can determine the format of the log entires which is:

   timestamp | time elapsed | remotehost | code/status | bytes | method | URL rfc931 peerstatus/peerhost | type

Step

...

3: Create a Grok Statement to Parse the Squid Telemetry Event

Now we are ready to tackle the Metron parsing topology setup.

...