Page History

Now that we have added the Squid Data Source, we want to visualize the data in the Metron Dashboard. The below provides instructions. 

Step 1: Setup and Pre-requisites

1. You should have completed the instructions in Adding a new Telemetry Data Source
2. Make sure the following variables are configured based on your environment: 

   
   KAFKA_HOST = host where a Kafka broker is installed
   ZOOKEEPER_HOST = host where a Zookeeper server is installed
   PROBE_HOST = Host where your sensor, probes are installed. If don't have any sensors installed, pick the host where a storm supervisor is running
   SQUID_HOST = Host where you want to install SQUID. If you don't care, just install on the PROBE_HOST
   NIFI_HOST = The host where you will install NIFI. You want this this to be same host that you installed Squid.
   HOST_WITH_ENRICHMENT_TAG = This is the host in your inventory hosts file that you put under the group "enrichment" 
   SEARCH_HOST = This is the host where you have elastic or solr running. This is the host in your inventory hosts file that you put under the group "search". Pick one of the search hosts
   SEARCH_HOST_PORT = The port of the search host where indexing is configured. (e.g: 9300)
   METRON_UI_HOST = This is the host where your metron ui web application is running. This is the host in your inventory hosts file that you put under the group "web".
   METRON_VERSION = The release of the metron binaries you are working with (e.g: 0.2.0BETA-RC2)

Creating Your Own Dashboard

Now that you understand Metron's default dashboard, let's cover how you might extend this dashboard for your own purposes. We will continue the ongoing example of parsing Squid Proxy logs. The dashboard will be extended to display the Squid log data.

Enhance the Squid Data

The previous tutorials covering Squid produced a limited data set. These consisted of a few basic requests. To make this tutorial more interesting, we are going to need a bit more variety in the sample data.

...

4. Ensure that the parser topology for Squid continues to run based on the steps outlined in the previous tutorials.

Create an Index Template

To work with the Squid data in Kibana, we need to ensure that the data is landing in the search index with the correct data types. This can be achieved by defining an index template.

...

    curl -XGET node1:9200/squid*

Configure the Squid Index in Kibana

Now that we have a Squid index with all of the right data types, we need to tell Kibana about this index.

...

4. Then click the 'Create' button.

Review the Squid Data

Now that Kibana is aware of the new Squid index, let's take a look at the data.

...

3. Clicking on a specific record will show each field available in the data.

Save a Squid Search

Let's create a basic data table so that a user can inspect record-level details for Squid.  In Kibana, this is done by creating a 'Saved Search'

...

3. Click on the 'Save' icon near the top-right to save the search.

Visualize the Squid Data

After using the `Discover` panel to better understand the Squid data, let's create a few visualizations.

...

6. Near the top-right side of the screen click on the 'Save' icon to save the visualization. Name it something appropriate. This will allow us to use the visualization in a dashboard later.

Customize the Dashboard

...

5. Continue enhancing the dashboard by adding the 'Saved Search' that was previously created.

Summary

At this point you should be comfortable customizing a dashboard as you add new sources of telemetry to Metron. This article introduced Metron's default dashboard that is built upon Kibana 4. It covered the elements present in the dashboard and how you can extend the dashboard for your own purposes.