Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

If you are running a version of ElasticSearch that is different from what came with Apache Metron 0.4.2 and older, the below information is outdated.  As of METRON-939, ElasticSearch and Kibana have been upgraded and the below guide may no longer be accurate.


Sizing your hardware

Elasticsearch

...

prefers to run on a large number of small servers, as opposed to multiple instances on big servers.  See this great post for more details.

If you are going to run multiple instance on a single server, be sure to provided dedicated disks to every instance.

Elasticsearch

...

On installation
1. Make sure to update your ES templates to properly index the fields of what you're sending in, especially if your logs have any custom fields.   Note that `bro_index.template` only currently handles DNS and HTTP Bro logs.  In the future I may be able to contribute back a template that handles all of the standard Bro logs, but I just haven't gotten a chance to push that out yet.  

...

Post-installation but before data ingest (i.e. no easy method for Metron's installer to do this right now)
1.  (optional) Consider multiple ES nodes per physical server.  Lots of guides online about this, but tread carefully.  My default would be to not pursue this unless you have large(ish) machines running as dedicated search nodes.  As far as I'm aware, Metron has no built-in way to provision this, and there may be some downstream .

...

TODO - However, most of the Elasticsearch tuning applies because they both use Lucene underneath, just different implementation methods...