Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: update hive.fetch.task.conversion & hive.fetch.task.conversion.threshold defaults (HIVE-7397)

...

hive.fetch.task.conversion
  • Default Value: minimal in Hive 0.10.0 through 0.13.1, more in Hive 0.14.0 and later
  • Added In: Hive 0.10.0 with HIVE-2925; default changed in Hive 0.14.0 with HIVE-7397

Some select queries can be converted to a single FETCH task, minimizing latency. Currently the query should be single sourced not having any subquery and should not have any aggregations or distincts (which incur RS – ReduceSinkOperator, requiring a MapReduce task), lateral views and joins.

...

hive.fetch.task.conversion.threshold
  • Default Value: -1 in Hive 0.13.0 and 0.13.11073741824 (1 GB) in Hive 0.14.0 and later 
  • Added In: Hive 0.13.0 with HIVE-3990; default changed in Hive 0.14.0 with HIVE-7397

Input threshold (in bytes) for applying hive.fetch.task.conversion. If target table is native, input length is calculated by summation of file lengths. If it's not native, the storage handler for the table can optionally implement the org.apache.hadoop.hive.ql.metadata.InputEstimator interface. A negative threshold means hive.fetch.task.conversion is applied without any input length threshold.

...