...
| Code Block |
|---|
|
wget https://dlcdn.apache.org/tez/0.10.2/apache-tez-0.10.2-bin.tar.gz
tar -xzvf apache-tez-0.10.2-bin.tar.gz
cd apache-tez-0.10.2-bin
tar zcvf ../apache-tez-0.10.2-bin.tar.gz * && cd .. |
Replace hdfs-client jar to 3.3.6 (tez is shipped with 3.3.1)
| No Format |
|---|
rm apache-tez-0.10.2-bin/lib/hadoop-hdfs-client-3.3.1.jar
cp $HADOOP_HOME/share/hadoop/hdfs/hadoop-hdfs-client-3.3.6.jar $TEZ_HOME/lib |
Add the necessary tez files to hdfs
| Code Block |
|---|
|
$HADOOP_HOME/bin/hadoop fs -mkdir -p /apps/tez
$HADOOP_HOME/bin/hadoop fs -put apache-tez-0.10.2-bin.tar.gz /apps/tez # copy the tarball
$HADOOP_HOME/bin/hadoop fs -put apache-tez-0.10.2-bin /apps/tez # copy the whole folder |
Set up TEZ_HOME and HADOOP_CLASSPATH environment variablevariables
| Code Block |
|---|
|
export TEZ_HOME=/Users/zsoltmiskolczi/work/hive/hive-from-tar/apache-tez-0.10.2-bin
export HADOOP_CLASSPATH=$TEZ_HOME/*:$TEZ_HOME/conf |
Create a new config file for Tez: $TEZ_HOME/conf/tez-site.xml
| Code Block |
|---|
|
<configuration>
<property>
<name>tez.lib.uris</name>
<value>hdfs://localhost:9000/apps/tez/apache-tez-0.10.2-bin.tar.gz,hdfs://localhost:9000/apps/tez/apache-tez-0.10.2-bin/lib,hdfs://localhost:9000/apps/tez/apache-tez-0.10.2-bin</value>
</property>
<property>
<name>tez.use.cluster.hadoop-libs</name>
<value>true</value>
</property>
</configuration> |
Installing Hive from a Tarball
...
Next you need to unpack the tarball. This will result in the creation of a subdirectory named hive-x.y.z (where x.y.z is the release number):
| No Format |
|---|
$ wget https://dlcdn.apache.org/hive/hive-4.0.0-beta-1/apache-hive-4.0.0-beta-1-bin.tar.gz
tar -xzvf apache-hive-x4.y.z0.0-beta-1-bin.tar.gz
|
Set the environment variable HIVE_HOME to point to the installation directory:
| No Format |
|---|
$ cd apache-hive-x4.y.z
$ 0.0-beta-1-bin
export HIVE_HOME={{pwd}}
/Users/zsoltmiskolczi/work/hive/hive-from-tar/apache-hive-4.0.0-beta-1-bin |
Add Finally, add $HIVE_HOME/bin to your PATH:
| No Format |
|---|
$ export PATH=$HIVE_HOME/bin:$PATH |
Create a directory for external tables:
| No Format |
|---|
mkdir /yourpathtoexternaltables/warehouse |
Create a new config file for Hive: $HIVE_HOME/conf/hive-site.xml
| Code Block |
|---|
|
<configuration>
<property>
<name>hive.tez.container.size</name>
<value>1024</value>
</property>
<property>
<name>hive.metastore.warehouse.external.dir</name>
<value>/Users/zsoltmiskolczi/work/hive/hive-from-tar/warehouse</value>
</property>
<property>
<name>hive.execution.engine</name>
<value>tez</value>
</property>
<property>
<name>tez.lib.uris</name>
<value>hdfs://localhost:9000/apps/tez/apache-tez-0.10.2-bin.tar.gz,hdfs://localhost:9000/apps/tez/apache-tez-0.10.2-bin/lib,hdfs://localhost:9000/apps/tez/apache-tez-0.10.2-bin</value>
</property>
<property>
<name>tez.configuration</name>
<value>/Users/zsoltmiskolczi/work/hive/hive-from-tar/apache-tez-0.10.2-bin/conf/tez-site.xml</value>
</property>
<property>
<name>tez.use.cluster.hadoop-libs</name>
<value>true</value>
</property>
</configuration> |
Initialize metastore schema. It will create a directore called metastore_db. It contains an embedded Derby database for metastore
| Code Block |
|---|
|
$HIVE_HOME/bin/schematool -dbType derby -initSchema --verbose |
Run HiveServer2
| Code Block |
|---|
|
$HIVE_HOME/bin/hiveserver2 & |
Run beeline:
| Code Block |
|---|
|
bin/beeline -u 'jdbc:hive2://localhost:10000/' -n invisbleprogrammer |
Installing from Source Code
...