Page History

...

Apache Spark was added in Hive 1.1.0 (HIVE-7292 and the merge-to-trunk JIRA's HIVE-9257, 9352, 9448). For information see the design document Hive on Spark and Hive on Spark: Getting Started. 

To configure Hive execution to Spark, set the following property to "spark":

• hive.execution.engine

hive.spark.job.monitor.timeout

...

• Default Value: false
• Added In: Hive 1.3.0 with HIVE-9152

Turn When true, this turns on dynamic partition pruning for the Spark engine, so that joins on partition keys will be processed by writing to a temporary HDFS file, and read later for removing unnecessary partitions.

hive.spark.dynamic.partition.pruning.max.data.size

...