Page History

...

With the RecordReader initialized, its time to get some actual records! Pig calls HCatBaseLoader.getNext which gets an HCatRecord from the HCatRecordReader we just initialized, converts to a Pig tuple, and hands off to Pig for processing.

Let's explore how HCatRecord's are created. First, HCatRecordReader.nextKeyValue is called to get the actual record from the wrapped input format we created earlier. The record is first deserialized with the SerDe defined for the partition, and wrapped in a LazyHCatRecord, which delays further deserialization until required. Using the output schema set earlier, we create an HCatRecord with just the necessary fields. Finally, the HCatRecord is converted into a Pig tuple and handed off to Pig for processing.