Page History

...

Now that we know what we should do, the next question is how to accomplish it; in other words, we must define what exactly we mean when we say "severity."  The capability, as implemented in Metron, is accomplished by providing the ability to associate possibly complex conditions to numeric scores.  ThenThen, for each message, the set of conditions are evaluated and the set of numbers for matching conditions are aggregated via a configurable aggregation function.  This aggregated score is added to the message in the threat.triage.level.  Let's dig a bit deeper into this and provide an example.

...

The heart of the problem is how one defines we define a "condition."  In Metron, we provide a custom domain specific language for defining conditions.  

...

{

  ...

  ,"threatIntel" : {

            ...

           , "triageConfig" : {

                     "riskLevelRules" : {

                                 "condition1" : level1

                               , "condition2" : level2

                                  ...

                                        }

                     ,"aggregator" : "MAX"

                             }

                  }

}

• riskLevelRules correspond to the set of condition-to-numeric-level mappings that define the threat triage for this particular sensor. 
• aggregator is an aggregation function that takes all non-zero scores representing the matching queries from riskLevelRules and aggregates them into a single score.  The current supported aggregation functions are the following:
  • MAX : The max of all of the associated values for matching queries.
  • MIN : The min of all of the associated values for matching queries.
  • MEAN : The mean of all of the associated values for matching queries.
  • POSITIVE_MEAN : The mean of the positive associated values for the matching queries.

Step 1: Setup and

...

Prerequisites

1. Complete the instructions in Adding a new Telemetry Data Source.
2. Make sure the following variables are configured based on your environment: 

    

   • KAFKA_HOST = The host where a Kafka broker is installed.
   • ZOOKEEPER_HOST = The host where a Zookeeper server is installed.
   • PROBE_HOST = The host where your sensor, probes are installed. If don't have any sensors installed, pick the host where a Storm supervisor is running.
   • SQUID_HOST = The host where you want to install SQUID. If you don't care, just install SQUID on the PROBE_HOST.
   • NIFI_HOST = Host where you will install NIFI. You want this this to be same host on which you installed Squid.
   • HOST_WITH_ENRICHMENT_TAG = The host in your inventory hosts file that you put under the group "enrichment." 
   • SEARCH_HOST = The host where you have Elastic or Solr running. This is the host in your inventory hosts file that you put under the group "search". Pick one of the search hosts.
   • SEARCH_HOST_PORT  = The port of the search host where indexing is configured. (e.g., 9300)
   • METRON_UI_HOST = The host where your Metron UI web application is running. This is the host in your inventory hosts file that you put under the group "web."
   • METRON_VERSION = The release of the Metron binaries you are working with. (e.g., 0.2.0BETA-RC2)

...

1. Log into $HOST_WITH_ENRICHMENT_TAG as root.

2. We need to modify /usr/metron/$METRON_RELEASE/config/zookeeper/sensors/squid.json. However, since the configuration in Zookeeper may be out of sync with the configuration on disk, we must make sure they are in sync by downloading the Zookeeper configuration first:

   /usr/metron/$METRON_RELEASE/bin/zk_load_configs.sh -m PULL -z $ZOOKEEPER_HOST:2181 -f -o /usr/metron/$METRON_RELEASE/config/zookeeper

3. Validate that the the enrichment config for Squid exists.

   cat /usr/metron/$METRON_RELEASE/config/zookeeper/enrichments/squid.json

4. Edit the configuration.  In /usr/metron/$METRON_RELEASE/config/zookeeper/enrichments/squid.json add the following to the triageConfig section to the threat intel section.

   "threatIntel" : {

       "fieldMap" : {

         "hbaseThreatIntel" : [ "domain_without_subdomains" ]

       },

       "fieldToTypeMap" : {

         "domain_without_subdomains" : [ "zeusList" ]

       },

       "config" : { },

       "triageConfig" : {

         "riskLevelRules" : {

            "exists(threatintels.hbaseThreatIntel.domain_without_subdomains.zeusList)" : 5

                 , "not(ENDS_WITH(domain_without_subdomains, '.com') or ENDS_WITH(domain_without_subdomains, '.net'))" : 10

                              }

           ,"aggregator" : "MAX"

           ,"aggregationConfig" : { }

                         }

                     }

     }

5. Ensure that the aggregator field indicates MAX.

6. After modifying the configuration, we can push the configuration back to Zookeeper and have the enrichment topology pick it up with live data by running the following:

   /usr/metron/$METRON_RELEASE/bin/zk_load_configs.sh -m PUSH -z $ZOOKEEPER_HOST:2181 -i /usr/metron/$METRON_RELEASE/config/zookeeper zookeeper 

Step 4: View Triaged/Scored Alerts

...