...

Query and DDL Execution

hive.execution.engine

...

• Default Value: true in Hive 0.13.0 and 0.13.1; false in Hive 0.14.0 and later (HIVE-8151)
• Added In: Hive 0.13.0 with HIVE-6455
• Deprecated: replaced with hive.optimize.sort.dynamic.partition.threshold
• Removed in Hive 4.0 with HIVE-25320

When enabled, dynamic partitioning column will be globally sorted. This way we can keep only one record writer open for each partition value in the reducer thereby reducing the memory pressure on reducers.

...

A comma separated list of hooks which implement QueryLifeTimeHook. These will be triggered before/after query compilation and before/after query execution, in the order specified. As of Hive 3.0.0 (HIVE-16363), this config can be used to specify implementations of QueryLifeTimeHookWithParseHooks. If they are specified then they will be invoked in the same places as QueryLifeTimeHooks and will be invoked during pre and post query parsing.

hive.remove.orderby.in.subquery

• Default Value: true
• Added In: Hive 3.0.0 with HIVE-6348

If set to true, order/sort by without limit in subqueries and views will be removed.

will be invoked in the same places as QueryLifeTimeHooks and will be invoked during pre and post query parsing.

hive.remove.orderby.in.subquery

• Default Value: true
• Added In: Hive 3.0.0 with HIVE-6348

If set to true, order/sort by without limit in subqueries and views will be removed.

Datetime

hive.datetime.formatter

• Default Value: DATETIME
• Added In: Hive 4.0.0 with

  Jira
  server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key HIVE-25576

  ,

  Jira
  server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key HIVE-27673

The formatter to use for handling datetime values. The possible values are:

• DATETIME: For using java.time.format.DateTimeFormatter
• SIMPLE: For using java.text.SimpleDateFormat (known bugs: HIVE-25458, HIVE-25403, HIVE-25268)

hive.datetime.formatter.resolver.style

• Default Value: SMART
• Added in: Hive 4.0.0 with HIVE-27772 

The style used by the hive.datetime.formatter (only applicable to DATETIME) to resolve dates amd times. The possible values are:

• SMART:
  • Using smart resolution will perform the sensible default for each field, which may be the same as strict, the same as lenient, or a third behavior. Individual fields will interpret this differently.
  • For example, resolving year-month and day-of-month in the ISO calendar system using smart mode will ensure that the day-of-month is from 1 to 31, converting any value beyond the last valid day-of-month to be the last valid day-of-month.
• STRICT:
  • Using strict resolution will ensure that all parsed values are within the outer range of valid values for the field. Individual fields may be further processed for strictness.
  • For example, resolving year-month and day-of-month in the ISO calendar system using strict mode will ensure that the day-of-month is valid for the year-month, rejecting invalid values.
  • When using Strict as the hive.datetime.formatter.resolver.style we should use the pattern "u" to represent year. For more details, please refer: https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html  
• LENIENT:
  • Lenient mode allows the month in the ISO calendar system to be outside the range 1 to 12. For example, month 15 is treated as being 3 months after month 12.
    

Currently these configuration only affects the behavior of the following SQL functions:

• unix_timestamp(string,[string])
• from_unixtime
• date_format

The SIMPLE formatter exists purely for compatibility purposes with previous versions of Hive thus its use is discouraged. It suffers from known bugs that are unlikely to be fixed in subsequent versions of the product. Furthermore, using SIMPLE formatter may lead to strange behavior, and unexpected results when combined with SQL functions/operators that are using the new DATETIME formatter.

SerDes and ISerDes and I/O

SerDes

hive.script.serde

...

Used to avoid all of the proxies and object copies in the metastore. Note, if this is set, you MUST use a local metastore (hive.metastore.uris must be empty) otherwise undefined and most likely undesired behavior will resultresult.

hive.metastore.jdbc.max.batch.size

• Default Value: 1000
• Added In: Hive 4.0.0 with HIVE-23093

This controls the maximum number of update/delete/insert queries in a single JDBC batch statement.

Hive Metastore Connection Pooling Configuration

The Hive Metastore supports several connection pooling implementations (e.g. hikaricp, bonecp, dbcp). Configuration properties prefixed by 'hikari ' or 'dbcp' in versions prior to Hive 4.0.0-alpha-1 will be propagated as is to the connectionpool connection pool implementation by Hive. Starting in release 4.0.0-alpha-1, when using hikaricp, properties prefixed by 'hikaricp' will be propagated to the underlying connection pool. Jdbc connection url, username, password and connection pool maximum connections are exceptions which must be configured with their special Hive Metastore configuration properties.

...

• hive.txn.manager
• hive.compactor.initiator.on
• hive.compactor.cleaner.on
• hive.compactor.worker.threads

...

Turning on Hive transactions also requires appropriate settings for hive.compactor.initiator.on , hive.compactor.cleaner.on, hive.compactor.worker.threads , hive.support.concurrency , hive.enforce.bucketing  (Hive 0.x and 1.x only), and hive.exec.dynamic.partition.mode .

...

Whether to run the initiator and cleaner threads thread on this metastore instance. Set this to true on one instance of the Thrift metastore service as part of turning on Hive transactions. For a complete list of parameters required for turning on transactions, see hive.txn.manager .

Before Hive 1.3.0 it's critical that this is enabled on exactly one metastore service instance. As of  Hive 1.3.0  this property may be enabled on any number of standalone metastore instances.

hive.compactor.cleaner.on

• Default Value: false
• Hive Transactions Value:  true  (for exactly one instance of the Thrift metastore service)
• Added In: Hive 4.0.0 with HIVE-26908

Whether to run the Cleaner thread on this metastore instance. Set this to true on one on one instance of the Thrift metastore service as service as part of turning on Hive transactions. For a complete list of parameters required for turning on transactions, see hive.txn.manager .It's critical that this is enabled on exactly one metastore service instance (not enforced yet).

Before Hive 4.0.0 Cleaner thread can be started/stopped with config hive.compactor.initiator.on. This config helps to enable/disable initiator/cleaner threads independently

hive.compactor.worker.threads

...

Number of aborted transactions involving a given table or partition that will trigger a major compaction.

hive.compactor.aborted.txn.time.threshold

• Default Value: 12h
• Added In: Hive 4.0.0 with HIVE-23280

Age of table/partition's oldest aborted transaction when compaction will be triggered.
Default time unit is: hours. Set to a negative number to disable.

Compaction History

hive.compactor.history.retention.succeeded 

...

Number of failed compaction entries to retain in history (per partition).

...

metastore.compactor.history.retention

...

.did.not.initiate

• Default Value: 2
• Added In: Hive 1.3.0 and 2.0.0 with HIVE-12353

...

• Deprecated name: hive.compactor.history.retention.attempted 

Determines how many compaction records in state 'did not initiate' will be retained in compaction history for a given table/partition.

hive.compactor.history.reaper.interval

...

Enable metrics on the Hive Metastore Service. (For other metastore configuration properties, see the Metastore and Hive Metastore Security sections.)

hive.metastore.acidmetrics.thread.on

• Default Value: true
• Added in: Hive 4.0.0 with HIVE-24824

Whether to run acid related metrics collection on this metastore instance.

hive.server2.metrics.enabled

...