THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
SqoopDestroyerOutputCommitter is a custom outputcommiter that provides hooks to do final cleanup or in some cases the one-time operations we want to invoke when sqoop job finishes, i,e either fails or succeeds.
SqoopReducer
Extends the Reducer API and at this point only runs the progressService. It is invoked only when the numLoaders driver config is > 0. See above. VB: It is still un clear to me, how this would support throttling as indicated in this ticket, Looking for some details on how invoking a reducer helps.
| Code Block |
|---|
public class SqoopReducer extends Reducer<SqoopWritable, NullWritable, SqoopWritable, NullWritable> {
..
progressService.scheduleAtFixedRate(new SqoopProgressRunnable(context), 0, 2, TimeUnit.MINUTES);
} |
...