Bigtop Packaging
One of the longer term goals is to have Bigtop packages serve as as reference to eventually get Hadoop introduced into most Linux distributions.
Currently, this is problematic owning to the lack of maven3 native packaging for many major Linux distributions. However, Bigtop makefiles and other artifacts, create an alterntive build root, which then uses native rpm or .deb packaging tools to create packages.
We strive to follow best practices with both rpm and Debian based distributions.
If you discover packaging or installation issues, please file a ticket https://issues.apache.org/jira/browse/BIGTOP
Getting started with bigtop packaging.
1) You will have to package both deb and rpm. Starting with one or the other is the best.
2) Here are some steps you can follow to build a new bigtop packagein.
- determine where your source is, and add it to bigtop.mk
- update the bigtop-packages/src/common/<your-package> folder to have your component, and the do-component-build for it (which usually just builds a jar). Why is there a "common" directory? Simply because deb and rpm packaging share some tasks (like
do-component-build
, which just usually runs a mvn or gradle command), and so we keep a common install directory which they can both leverage for packaging. - If you need to apply a patch to the upstream source:
- create a patch file (in the
patch -p1
syntax) and place it in the bigtop-packages/src/common/<your-package> common folder. Name should be patch*.diff - for RPM Support place a #BIGTOP_PATCH_FILES token into the section with the Sources and #BIGTOP_PATCH_COMMANDS between %setup and %build
- for DEB it is applied by default.
- multiple patch files are applied in the glob sorting order
- create a patch file (in the
- (for RPM) now add a .spec file into bigtop-packages/src/<your-package>/... into the appropriate directory (i.e. bigtop-packages/src/rpm/tachyon/SPECS/tachyon.spec). Obviously, your tachyon.spec file will use whats in common/ in a RPM specific way, to install the RPM package.
- (for Debian) same as above ^ but of course, follow the debuild debian idioms:
- create
rules
usingdo-component-build
- create
compat
, please use version 9 for new packages - create control, please use for new packages:
Build-Depends: debhelper (>= 9)
Standards-Version: 3.9.4
- create
- Finally add a smoke test! This is as easy as adding a new groovy file to bitop-tests/smoke-tests/<your-package>/TestThisStuff.groovy, following conventions that others have created.
3) As always, we will improve on the directions above, but this should help to get you started. .
Helpful docs on bigtop packaging (will expand this table over time).
Description | URL | |||||
---|---|---|---|---|---|---|
Romans talk on RPMs vs Tarballs and BigTop Packaging |
| |||||
Walkthrough of RPM spec for hadoop in BigTop | http://jayunit100.blogspot.com/2014/04/how-bigtop-packages-hadoop.html | |||||
RPM Macros | ||||||
RPM %Files | http://www.rpm.org/max-rpm-snapshot/s1-rpm-inside-files-list-directives.html |
|