| Table of Contents |
|---|
| maxLevel | 3 |
|---|
| outline | true |
|---|
| indent | 10px |
|---|
| style | squarecircle |
|---|
|
There are two ways to try out Ozone. Either you can build from source code or download a binary release.
...
| Code Block |
|---|
| language | bash |
|---|
| theme | RDark |
|---|
| collapse | true |
|---|
|
tar xf hadoop-ozone-0.3.0-alpha-src.tar.gz
cd hadoop-ozone-0.3.0-alpha-src-with-hdds/
mvn clean install -Phdds -DskipTests=true -Dmaven.javadoc.skip=true -Pdist -Dtar -DskipShade |
...
| Code Block |
|---|
|
mvn clean install -Phdds -DskipTests=true -Dmaven.javadoc.skip=true -Pdist -Dtar -DskipShade -am -pl :hadoop-ozone-dist |
...
| Code Block |
|---|
| language | bash |
|---|
| theme | RDark |
|---|
| collapse | true |
|---|
|
tar xf hadoop-ozone-0.3.0-alpha.tar.gz
cd hadoop-ozone-0.3.0-alpha/ |
Start Cluster Using Docker
If you downloaded or built a source release, run the following commands to start an Ozone cluster in docker containers with 3 datanodes.
| Code Block |
|---|
|
cd hadoop-ozone/dist/target/ozone-*-SNAPSHOT/compose/ozone
docker-compose up -d --scale datanode=3 |
If you downloaded a binary release, run the following instead.,
| Code Block |
|---|
|
cd compose/ozone
docker-compose up -d --scale datanode=3 |
For more docker-compose commands, please check the end of the Getting started with docker guide
To Shutdown the cluster, please run the command docker-compose down
Single Node Development Cluster
This is the traditional way to start a development cluster from source code. Once the package is built, you can start Ozone services by going to the hadoop-ozone/dist/target/ozone-*/ directory. Your Unix shell should expand the '*' wildcard to the correct Ozone version number.
Configuration
Save the minimal snippet to hadoop-ozone/dist/target/ozone-*/etc/hadoop/ozone-site.xml in the compiled distribution.
| Code Block |
|---|
|
<configuration>
<properties>
<property><name>ozone.enabled</name><value>true</value></property>
<property><name>ozone.scm.datanode.id</name><value>/tmp/ozone/data/datanode.id</value></property>
<property><name>ozone.replication</name><value>1</value></property>
<property><name>ozone.metadata.dirs</name><value>/tmp/ozone/data/metadata</value></property>
<property><name>ozone.scm.names</name><value>localhost</value></property>
<property><name>ozone.om.address</name><value>localhost</value></property>
</properties>
</configuration> |
Start Services
To start ozone, you need to start SCM, OzoneManager and DataNode. In pseudo-cluster mode, all services will be started on localhost.
| Code Block |
|---|
|
bin/ozone scm --init
bin/ozone --daemon start scm
bin/ozone om --init
bin/ozone --daemon start om
bin/ozone --daemon start datanode |
Run Ozone Commands
Once you have ozone running you can use these Ozone shell commands to create a volume, bucket and keys. E.g.
| Code Block |
|---|
|
bin/ozone sh volume create /vol1
bin/ozone sh bucket create /vol1/bucket1
dd if=/dev/zero of=/tmp/myfile bs=1024 count=1
bin/ozone sh key put /vol1/bucket1/key1 /tmp/myfile
bin/ozone sh key list /vol1/bucket1 |
Stop Services
| Code Block |
|---|
|
bin/ozone --daemon stop om
bin/ozone --daemon stop scm
bin/ozone --daemon stop datanode |
Clean up your Dev Environment (Optional)
Remove the following directories to wipe the Ozone pseudo-cluster state. This will also delete all user data (volumes/buckets/keys) you added to the pseudo-cluster.
| Code Block |
|---|
|
rm -fr /tmp/ozone
rm -fr /tmp/hadoop-${USER}* |
| Info |
|---|
Note: This will also wipe state for any running HDFS services.
|
Multi-Node Ozone Cluster
Pre-requisites
Ensure you have password-less ssh setup between your hosts.
Configuration
ozone-site.xml
Save the following snippet to etc/hadoop/ozone-site.xml in the compiled Ozone distribution.
| Code Block |
|---|
|
<configuration>
<properties>
<property><name>ozone.scm.block.client.address</name><value>SCM-HOSTNAME</value></property>
<property><name>ozone.scm.names</name><value>SCM-HOSTNAME</value></property>
<property><name>ozone.scm.client.address</name><value>SCM-HOSTNAME</value></property>
<property><name>ozone.om.address</name><value>OM-HOSTNAME</value></property>
<property><name>ozone.handler.type</name><value>distributed</value></property>
<property><name>ozone.enabled</name><value>True</value></property>
<property><name>ozone.scm.datanode.id</name><value>/tmp/ozone/data/datanode.id</value></property>
<property><name>ozone.replication</name><value>1</value></property>
<property><name>ozone.metadata.dirs</name><value>/tmp/ozone/data/metadata</value></property>
</properties>
</configuration> |
Replace SCM-HOSTNAME and OM-HOSTNAME with the names of the machines where you want to start the SCM and OM services respectively. It is okay to start these services on the same host. If you are unsure then just use any machine from your cluster.
hadoop-env.sh
The only mandatory setting in hadoop-env.sh is JAVA_HOME. E.g.
| Code Block |
|---|
|
# The java implementation to use. By default, this environment
# variable is REQUIRED on ALL platforms except OS X!
export JAVA_HOME=/usr/java/latest |
workers
The workers file should contain a list of hostnames in your cluster where DataNode service will be started. E.g.
| Code Block |
|---|
|
n001.example.com
n002.example.com
n003.example.com
n004.example.com |
Start Services
Initialize the SCM
Run the following commands on the SCM host
| Code Block |
|---|
|
bin/ozone scm -init
bin/ozone --daemon start sum |
Format the OM
Run the following commands on the OM host
| Code Block |
|---|
|
bin/ozone om -createObjectStore
bin/ozone --daemon start om |
Start DataNodes
Run the following command on any cluster host.
| Code Block |
|---|
|
sbin/start-ozone.sh |
Stop Services
Run the following command on any cluster host.
| Code Block |
|---|
|
sbin/stop-ozone.sh |