Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: add hive.mapjoin.optimized.hashtable and hive.mapjoin.optimized.hashtable.wbsize (HIVE-6430)

...

Whether a MapJoin hashtable should use optimized (size-wise) keys, allowing the table to take less memory. Depending on the key, memory savings for the entire table can be 5-15% or so.

hive.mapjoin.optimized.hashtable
  • Default Value: true
  • Added In: Hive 0.14.0 with HIVE-6430 

Whether Hive should use a memory-optimized hash table for MapJoin. Only works on Tez, because memory-optimized hash table cannot be serialized.

hive.mapjoin.optimized.hashtable.wbsize
  • Default Value: 10485760 (10 * 1024 * 1024)
  • Added In: Hive 0.14.0 with HIVE-6430 

Optimized hashtable (see hive.mapjoin.optimized.hashtable) uses a chain of buffers to store data. This is one buffer size. Hashtable may be slightly faster if this is larger, but for small joins unnecessary memory will be allocated and then trimmed.

hive.mapjoin.lazy.hashtable

...

Besides the configuration properties listed in this section, some properties in other sections are also related to Tez:

...