Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Overall there are 2 ways to implementing this functionality using the KiteSDK

Option 1

Duplicate a lot of the code in KiteConnector and add a new independent connector for KiteHbaseConnector. The major con is the code duplication and effort to support Yet another connector

 

Option 2:

  • Use the current KiteConnector and add a enum to select the type of dataset Kite will create underneath, or parse to URI given in the FromJobConfig and ToJobConfig to figure out the dataset to be HIVE/ Hbase or HDFS

    Code Block
    public enum DataSetType {
      HDFS,
      HBASE,
      HIVE
    }
    // use this enum to determine what dataset kite needs to create underneath
      @Input
      public DataSetType datasetType
     
    or
    // parse this to figure out the data set
      @Input(size = 255, validators = {@Validator(DatasetURIValidator.class)})
    
      public String uri

...