You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

As part of completion of BIGTOP-996, we have created a list of HOW-TOs for bigtop, which will be maintained over time.  Please notify and open a JIRA if any of these become obsolete.

Create a BigTop Hadoop VM cluster: hadoop cluster  reproducible, easily destroyed/recreated, configurable hadoop sandbox cluster with vagrant

See bigtop-deploy/, and go into the README files underneath vagrant. 

 

Create a BigTop Hadoop VM : Static one node hadoop cluster by loading a VM directly

See bigtop-deploy/, and go into the README files underneath vagrant-puppet.


Use the BigPetStore application to smoke test or demonstrate a hadoop deployment

See bigtop-bigpetstore, and follow the README file. 

 

Run the smoke tests + API tests for Apache BigTop's hadoop distribution on HDFS

See the README.md file in the root of bigtop/.  It has basic directions for maven based smoke tests.  Also track the BIGTOP-1222 JIRA,

which aims to improve these.

 

Run BigTop Ecosystem interoperability smoke tests (proper subset of the full bigtop smoke tests).

This is the BIGTOP-1222 JIRA which is underway.  For now, you should run a subset of the bigtop maven tests, and

customize them by commenting out submodules as necessary from test-execution/pom.xml.


Add a package for my new hadoop ecosystem tool "XYZ" to bigtop.
This takes experience and understanding of RPM / deb packaging.  Sean Macrory is a commiter for bigtop and can help get started down this road.  Best place to start is to ping the mailing list.  For introductions to bigtop packaging you can also read some of the posts at https://blogs.apache.org/bigtop/.


Learn about Apache Hadoop using BigTop

This is the most exciting thing about bigtop - it is a great sandbox for learning how hadoop really works.  You can start by playing with the bigpetstore app on your local machine, which can run a whole hadoop workflow for you, creating data that you can visualize in the bigpetstore web app  .  Once you "get it", you can move on to spinning up your own hadoop cluster using the vagrant recipes, or else, by directly grabbing a bigtop image from the nightly builds on the jenkins server.  Finally, as you get more advanced, you can see how bigtop pulls down source from various hadoop projects, packages it into rpms, installs those rpms, and coordinates their configuration/startup with puppet.

  • No labels