Bigtop Packaging
One of the longer term goals is to have Bigtop packages serve as as reference to eventually get Hadoop introduced into most Linux distributions.
Currently, this is problematic owning to the lack of maven3 native packaging for many major Linux distributions. However, Bigtop makefiles and other artifacts, create an alterntive build root, which then uses native rpm or .deb packaging tools to create packages.
We strive to follow best practices with both rpm and Debian based distributions.
If you discover packaging or installation issues, please file a ticket https://issues.apache.org/jira/browse/BIGTOP
Getting started with bigtop packaging.
1) You will have to package both deb and rpm. Starting with one or the other is the best.
2) Here are some steps you can follow to build a new bigtop package in.
- determine where your source is, and add it to
bigtop.bom
- update the
bigtop-packages/src/common/<your-package>
folder to have your component, and thedo-component-build
for it (which usually just builds a jar). Why is there a "common" directory? Simply because deb and rpm packaging share some tasks (likedo-component-build
, which just usually runs a mvn or gradle command), and so we keep a common install directory which they can both leverage for packaging. - If you need to apply a patch to the upstream source (use the utmost caution, in general Bigtop doesn't patch upstreams):
- create a patch file (in the
patch -p1
syntax) and place it in thebigtop-packages/src/common/<your-package>
common folder. Name should bepatch*.diff
- for RPM Support place a #BIGTOP_PATCH_FILES token into the section with the
Sources
and #BIGTOP_PATCH_COMMANDS between%setup
and%build
- for DEB the patch is applied by default.
- multiple patch files are applied in the glob sorting order
- create a patch file (in the
- (for RPM) now add a
.spec
file intobigtop-packages/src/<your-package>/...
into the appropriate directory (i.e.bigtop-packages/src/rpm/tachyon/SPECS/tachyon.spec
). Obviously, yourtachyon.spec
file will use whats incommon
/ in a RPM specific way, to install the RPM package. - (for Debian) same as above ^ but of course, follow the
debuild
debian idioms:- create a
rules file
usingdo-component-build
- create a
compat
file, please use version 9 for new packages create a
control
file, please use for new packages:Build-Depends: debhelper (>= 9) and Standards-Version: 3.9.4
- create a
- Test it with
gradle <your-package>-deb
on Ubuntu/Debian or <your-package>-rpm for the others.- For DEB: Have a look at the lintian errors and warnings, please try to minimize the numbers.
- Finally add a smoke test! This is as easy as adding a new groovy file to bitop-tests/smoke-tests/<your-package>/TestThisStuff.groovy, following conventions that others have created.
3) As always, we will improve on the directions above, but this should help to get you started. .
Helpful docs on bigtop packaging (will expand this table over time).
Description | URL | |
---|---|---|
Romans talk on RPMs vs Tarballs and BigTop Packaging | ||
Walkthrough of RPM spec for hadoop in BigTop | http://jayunit100.blogspot.com/2014/04/how-bigtop-packages-hadoop.html | |
RPM Macros | ||
RPM %Files | http://www.rpm.org/max-rpm-snapshot/s1-rpm-inside-files-list-directives.html |
|
Debian Policies | https://www.debian.org/doc/debian-policy/ |