Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Further, in our design, metastore will read all metastore objects once at startup time and there is no eviction of the metastore objects ever since. The only time we change cache is when user requested a change through metastore client (eg, alter table, alter partition), and upon receiving metastore event of changes made by other metastore server.

Currently, the size of the metastore cache can be restricted by a combination of cache whitelist and blacklist patterns. Before a table is cached, it is checked against these filters to decide if it can be cached or not. Similarly, when a table is read, if it does not pass the above filters, it is read from the database and not the cache.

Quantitative study: memory footprint and prewarm time

...