Versions Compared

Old Version 6

changes.mady.by.user Lars Francke

Saved on

compared with

New Version 7

changes.mady.by.user Lars Francke

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Document HIVE-2822

...

  • Using the set command in the cli for setting session level values for the configuration variable for all statements subsequent to the set command. e.g.
    Code Block
      set hive.exec.scratchdir=/tmp/mydir;
    sets the scratch directory (which is used by hive to store temporary output and plans) to /tmp/mydir for all subseq
  • Using -hiveconf option on the cli for the entire session. e.g.
    Code Block
      bin/hive -hiveconf hive.exec.scratchdir=/tmp/mydir
  • In hive-site.xml. This is used for setting values for the entire Hive configuration. e.g.
    Code Block
    xml
    xml
      <property>
    <name>hive.exec.scratchdir</name>
    <value>/tmp/mydir</value>
    <description>Scratch space for Hive jobs</description>
  </property>

hive-default.xml

...

.template contains the default values for various configuration variables that come with prepackaged in a Hive distribution.

...

In order to override any of the values, create hive-site.xml instead and set the value in that file as shown above. Please note that this file is not used by Hive at all (as of Hive 0.9.0) and so it might be out of date or out of sync with the actual values. The canonical list of configuration options is now only managed in the HiveConf java class.

hive-default.xml.template is located in the conf directory in your installation root. hive-site.xml should also be created in the same directory.

...

Variable Name

Description

Default Value

hive.ddl.output.format

The data format to use for DDL output (e.g. DESCRIBE table). One of "text" (for human readable text) or "json" (for a json object). (as of Hive 0.9.0)

text

hive.exec.script.wrapper

Wrapper around any invocations to script operator e.g. if this is set to python, the script passed to the script operator will be invoked as python <script command>. If the value is null or not set, the script is invoked as <script command>.

null

hive.exec.plan

 

null

hive.exec.scratchdir

This directory is used by hive to store the plans for different map/reduce stages for the query as well as to stored the intermediate outputs of these stages.

/tmp/<user.name>/hive

hive.querylog.location

Directory where structured hive query logs are created. One file per session is created in this directory. If this variable set to empty string structured log will not be created.

/tmp/<user.name>

hive.exec.submitviachild

Determines whether the map/reduce jobs should be submitted through a separate jvm in the non local mode.

false - By default jobs are submitted through the same jvm as the compiler

hive.exec.script.maxerrsize

Maximum number of serialization errors allowed in a user script invoked through TRANSFORM or MAP or REDUCE constructs.

100000

hive.exec.compress.output

Determines whether the output of the final map/reduce job in a query is compressed or not.

false

hive.exec.compress.intermediate

Determines whether the output of the intermediate map/reduce jobs in a query is compressed or not.

false

hive.jar.path

The location of hive_cli.jar that is used when submitting jobs in a separate jvm.

 

hive.aux.jars.path

The location of the plugin jars that contain implementations of user defined functions and serdes.

 

hive.partition.pruning

A strict value for this variable indicates that an error is thrown by the compiler in case no partition predicate is provided on a partitioned table. This is used to protect against a user inadvertently issuing a query against all the partitions of the table.

nonstrict

hive.map.aggr

Determines whether the map side aggregation is on or not.

true

hive.join.emit.interval

 

1000

hive.map.aggr.hash.percentmemory

 

(float)0.5

hive.default.fileformat

Default file format for CREATE TABLE statement. Options are TextFile, SequenceFile and RCFile

TextFile

hive.merge.mapfiles

Merge small files at the end of a map-only job.

true

hive.merge.mapredfiles

Merge small files at the end of a map-reduce job.

false

hive.merge.size.per.task

Size of merged files at the end of the job.

256000000

hive.merge.smallfiles.avgsize

When the average output file size of a job is less than this number, Hive will start an additional map-reduce job to merge the output files into bigger files. This is only done for map-only jobs if hive.merge.mapfiles is true, and for map-reduce jobs if hive.merge.mapredfiles is true.

16000000

hive.enforce.bucketing

If enabled, enforces inserts into bucketed tables to also be bucketed

false

...