Page History

...

1. The proxy events from Squid logs needs to must be ingested in real-time.
2. The proxy logs has to must be parsed into a standardized JSON structure that Metron can understand.
3. In real-time, the squid proxy event needs to must be enriched so that the domain named are enriched with the IP information
4. In real-time, the IP with in the proxy event must be checked against for threat intel feeds.
5. If there is a threat intel hit, an alert needs to must be raised.
6. The end user must be able to see the new telemetry events and the alerts from the new data source.
7. All of this requirements will need to must be implemented easily without writing any new java Java code.

In this article, we will walk you through how to do meet requirements 4 and 5.

Threat Intel Framework Explained

Metron currently provides an extensible framework to plug in threat intel sources. Each threat intel source has two components: an enrichment data source and and enrichment bolt. The threat intelligence feeds are bulk loaded and streamed into a threat intelligence store similarly similar to how the enrichment feeds are loaded. The keys are loaded in a key-value format. The key is the indicator and the value is the JSON formatted description of what the indicator is. It is recommended to use a threat feed aggregator such as Soltra to dedup and normalize the feeds via Stix/Taxii. Metron provides an adapter that is able to read Soltra-produced Stix/Taxii feeds and stream them into HbaseHBase, which is the data store of choice to back high speed threat intel lookups of Metron. Metron additionally provides a flat file and Stix bulk loader that can normalize, dedup, and bulk load or stream threat intel data into Hbase even without the use of a threat feed aggregator.

...