THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
| Code Block | ||
|---|---|---|
| ||
public class HoodieSparkIndex<T extends HoodieRecordPayload>
extends HoodieGeneralIndex<
T,
JavaRDD<HoodieRecord<T>>,
JavaRDD<WriteStatus>,
JavaRDD<HoodieKey>,
JavaPairRDD<HoodieKey, Option<Pair<String, String>>>> {
@Override
public JavaPairRDD<HoodieKey, Option<Pair<String, String>>> fetchRecordLocation(
JavaRDD<HoodieKey> hoodieKeys, HoodieTable<T> hoodieTable) {
//...
}
@Override
public JavaRDD<HoodieRecord<T>> tagLocation(JavaRDD<HoodieRecord<T>> recordRDD,
HoodieTable<T> hoodieTable) throws HoodieIndexException {
//...
}
@Override
public JavaRDD<WriteStatus> updateLocation(JavaRDD<WriteStatus> writeStatusRDD,
HoodieTable<T> hoodieTable) throws HoodieIndexException {
//...
}
//...
} |
For the basic index class in Flink context, we can create a HoodieFlinkIndex, the definition is:
| Code Block | ||
|---|---|---|
| ||
public class HoodieFlinkIndex<T extends HoodieRecordPayload>
extends HoodieGeneralIndex<
T,
DataStream<HoodieRecord<T>>,
DataStream<WriteStatus>,
DataStream<HoodieKey>,
DataStream<Tuple2<HoodieKey, Option<Tuple2<String, String>>>>> {
public HoodieFlinkIndex(HoodieWriteConfig config) {
super(config);
}
@Override
public DataStream<Tuple2<HoodieKey, Option<Tuple2<String, String>>>> fetchRecordLocation(
DataStream<HoodieKey> hoodieKeys, HoodieTable<T> hoodieTable) {
return null;
}
@Override
public DataStream<HoodieRecord<T>> tagLocation(DataStream<HoodieRecord<T>> recordRDD,
HoodieTable<T> hoodieTable) throws HoodieIndexException {
return null;
}
@Override
public DataStream<WriteStatus> updateLocation(DataStream<WriteStatus> writeStatusRDD,
HoodieTable<T> hoodieTable) throws HoodieIndexException {
return null;
}
//...
} |
Rollout/Adoption Plan
None
...