Page History

Now that we have added the Squid Data Source, we want to visualize the data in the Metron Dashboard. The below provides instructions. 

Step 1: Setup and Pre-requisites

1. You should have completed the instructions in Adding a new Telemetry Data Source
2. Make sure the following variables are configured based on your environment: 

   
   KAFKA_HOST = host where a Kafka broker is installed
   ZOOKEEPER_HOST = host where a Zookeeper server is installed
   PROBE_HOST = Host where your sensor, probes are installed. If don't have any sensors installed, pick the host where a storm supervisor is running
   SQUID_HOST = Host where you want to install SQUID. If you don't care, just install on the PROBE_HOST
   NIFI_HOST = The host where you will install NIFI. You want this this to be same host that you installed Squid.
   HOST_WITH_ENRICHMENT_TAG = This is the host in your inventory hosts file that you put under the group "enrichment" 
   SEARCH_HOST = This is the host where you have elastic or solr running. This is the host in your inventory hosts file that you put under the group "search". Pick one of the search hosts
   SEARCH_HOST_PORT = The port of the search host where indexing is configured. (e.g: 9300)
   METRON_UI_HOST = This is the host where your metron ui web application is running. This is the host in your inventory hosts file that you put under the group "web".
   METRON_VERSION = The release of the metron binaries you are working with (e.g: 0.2.0BETA-RC2)

Metron's Dashboard

Metron's default dashboard is intended to allow you to easily validate the end-to-end functioning of Metron with its default sensor suite. It highlights some of the useful widgets available in Kibana 4, and serves as a starting point for you to build your own customized dashboards.

Image Removed

The first panel in the dashboard highlights the variety of events being consumed by Metron. It shows the total number of events received, the variety of those events, and a histogram showing when the events were received.

Image Removed

The next set of dashboard panels shows how Apache Metron can be used to perform real-time enrichment of telemetry data. All of the IPv4 data received by Metron was cross-referenced against a geo-ip database. These locations were then used to build this set of dashboard widgets.

Image Removed

As part of the default sensor suite, YAF is used to generate flow records. These flow records provide significant visibility into which actors are communicating over the target network. A table widget displays the raw details of each flow record. A histogram of the duration of each flow shows that while most flows are relatively short-lived there are a few that are exceptionally longer in this example. Creating an index template that defined this field as numeric was required to generate the histogram.

Image Removed

Snort is a Network Intrusion Detection System (NIDS) that is being used to generate alerts identifying known bad events. Snort relies on a fixed set of rules that act as signatures for identifying abnormal events. Along with displaying the relevant details of each alert, the panel shows that there is only a single unique alert type; a test rule that creates a Snort alert on every network packet. Another table was created to show source/destination pairs that generated the most Snort alerts.

Image Removed

The Bro Network Security Monitor is extracting application-level information from raw network packets. In this example, Bro is extracting HTTP and HTTPS requests being made over the network. The panels highlight the breakdown by request type, the total number of web requests, and raw details from each web request.

Image Removed

...

Creating Your Own Dashboard

...

Now that we have a Squid index with all of the right data types, we need to tell Kibana about this index.

...