Please consult with the following page to run a new development cluster based on your own build:

Development cluster with docker

And please check the official documentation to run your cluster from the latest available distribution.

https://hadoop.apache.org/ozone/docs/0.3.0-alpha/dozone.html


  • No labels

8 Comments

  1. Thanks for nice write up.. I was not able install successfully,looks something I missed..Below the snapshot for same.

    and OzoneConfiguration and simple script  links are not working.


    root@brahma:/home/brahma/trunk/hadoop/hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/compose/ozone# ozone freon randomkeys --numOfVolumes=10 --numOfBuckets 10 --numOfKeys 10 --replicationType=RATIS --factor=THREE
    No command 'ozone' found, did you mean:
    Command 'iozone' from package 'iozone3' (multiverse)
    ozone: command not found
    root@brahma:/home/brahma/trunk/hadoop/hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/compose/ozone# docker-compose ps
    root@brahma:/home/brahma/trunk/hadoop/hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/compose/ozone#


    OS:

    Linux brahma 4.13.0-39-generic #44~16.04.1-Ubuntu SMP Thu Apr 5 16:43:10 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

  2. Can you run this command from inside datanode instance. That is after the

    > docker-compose exec datanode bash

    if you are inside the datanode, the ozone shell command will be in path.

    Else, ozone command will in the bin directory of ozone, just like Hadoop.

    You can execute the ozone command from that location too. Ozone is a shell wrapper just like hdfs command.


    1. Thanks for prompt reply.

      Yes, I executed after "docker-compose exec datanode bash" only.

      As " docker-compose ps" doesn't give any instance list, I think some thing I missed..let me try like below comment as you suggested.

  3. Also we have some prepackaged ozone docker images in the tree now; so you can do a

    > mvn clean install -T 1C -Phdds -DskipTests=true -Dmaven.javadoc.skip=true -DskipShade

    and then you can go to the dist directory to find all kinds of docker images, the most simple one  is

    > /hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/compose/ozone


  4. I still dn't have the luck.anything else I am missing, sorry to trouble you.

    root@brahma:/home/brahma/trunk/hadoop/hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/compose/ozone# mvn clean install -T 1C -Phdds -DskipTests=true -Dmaven.javadoc.skip=true -DskipShade

    .......................................................................................

    ......................................................................................

    [INFO] Apache Hadoop Ozone FileSystem Legacy Jar Library .. SUCCESS [01:41 min]
    [INFO] Apache Hadoop Ozone Tools .......................... SUCCESS [ 28.121 s]
    [INFO] Apache Hadoop Ozone Datanode ....................... SUCCESS [ 18.291 s]
    [INFO] Apache Hadoop Ozone Distribution ................... SUCCESS [ 29.169 s]
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 07:41 min (Wall Clock)
    [INFO] Finished at: 2019-02-18T03:22:14+05:30
    [INFO] Final Memory: 509M/1684M
    [INFO] ------------------------------------------------------------------------
    root@brahma:/home/brahma/trunk/hadoop# cd hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/compose/ozone
    root@brahma:/home/brahma/trunk/hadoop/hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/compose/ozone# ls
    docker-compose.yaml docker-config
    root@brahma:/home/brahma/trunk/hadoop/hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/compose/ozone# docker
    docker docker-containerd docker-containerd-shim docker-init docker-runc
    docker-compose docker-containerd-ctr dockerd docker-proxy
    root@brahma:/home/brahma/trunk/hadoop/hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/compose/ozone# docker-compose up -d
    root@brahma:/home/brahma/trunk/hadoop/hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/compose/ozone# docker-compose up -d
    root@brahma:/home/brahma/trunk/hadoop/hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/compose/ozone# docker-compose ps
    root@brahma:/home/brahma/trunk/hadoop/hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/compose/ozone# docker-compose logs

  5. So what is the output of 'docker-compose logs' command? Do you see any error there?


  6. Looks problem docker-compose.. I re-installed..Now it's ok.. Thanks guys.


    Digest: sha256:0f74e5f67294a0fb3a7337f314df268c9c724c3059fef85f146494ed735c9e1b
    Status: Downloaded newer image for apache/hadoop-runner:latest
    Creating ozone_scm_1 ... done
    Creating ozone_datanode_1 ... done
    Creating ozone_ozoneManager_1 ... done
    root@brahma:/home/brahma/trunk/hadoop/hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/compose/ozone# docker-compose ps
    Name Command State Ports
    ----------------------------------------------------------------------
    ozone_datanode_1 /usr/local/bin/dumb-init - ... Exit 1
    ozone_ozoneManager_1 /usr/local/bin/dumb-init - ... Exit 1
    ozone_scm_1 /usr/local/bin/dumb-init - ... Exit 1

  7. I am deleting this page, the development version is documented here:

    Development cluster with docker

    The distribution version is documented here:

    https://hadoop.apache.org/ozone/docs/0.3.0-alpha/dozone.html