Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Metron Streaming is a module that deals with stream processing including telemetry ingest, enrichment, threat intelligence referencing, alerting, and real-time scoring of machine learning models.  Metron Streaming is built on top of Apache Storm, which is a massively scalable stream processing engine with unique properties that make it a good fit for processing networking and logging data.  Telemetry generated by various sensors is processed by Metron's Storm topologies.  There are three topology types.  The first type is a PCAP topology, which is a topology dedicated to capturing and storing network packets.  The  

 

View file
nameTopologyOverview.pptx
height400

Topology NameDescriptionArchitecture Reference
Parsing/Normalizing TopologyReceives a telemetry message in it's native format and normalizes it to a common Metron JSON format. There is one topology per source and the output is piped to the Enrichment/Threat Intel topologyParsing Topology
Enrichment/Threat Intel TopologyTakes an normalized Metron JSON, enriches it, cross-references it against threat intelligence, tags it with alerts (where appropriate), runs the result against the scoring component of machine learning models (where appropriate) and stores the telemetry in a data store supported by MetronEnrichment/Threat Intel Topology
PCAP TopologyThe PCAP topology is designed to process telemetry produced

...

...

 and it's output is designed to be visualized

...