You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

In this blog post we will walk through what it takes to setup a new telemetry source in Metron.  For this example we will setup a new sensor, capture the sensor logs, pipe the logs to Kafka, pick up the logs with a Metron parsing topology, parse them, and run them through the Metron stream processing pipeline.  

Our example sensor will be a Squid Proxy.  Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP, and more.  Squid logs are simple to explain and easy to parse and the velocity of traffic coming from Squid is representative of a a typical network-based sensor.  Hence, we feel it's a good telemetry to use for this tutorial.

 

Prior to going through this tutorial make sure you have Metron properly installed.  Please see here for Metron installation and validation instructions.  We will be using a single VM setup for this exercise.  To setup the VM do the following steps:

 

cd deployment/vagrant/singlenode-vagrant
vagrant plugin install vagrant-hostmanager
vagrant up
vagrant ssh

After executing the above commands a Metron VM will be build (called node1) and you will be logged in as user vagrant.  Now lets install the Squid sensor.  

sudo yum install squid

sudo service squid start 

This will run through the install and the Squid sensor will be installed and started.  Now lets look at Squid logs.

sudo su -

cd /var/log/squid

ls 

You see that there are three types of logs available: access.log, cache.log, and squid.out.  We are interested in access.log as that is the log that records the proxy usage.  We see that initially the log is empty.  Lets generate a few entries for the log.

vi /etc/squid/squid.conf

 

 

 

 

 

 

 

  • No labels