Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

...

  • Sqoop schema is mandated, since we need a schema to construct a avro record

    Code Block
     // convert the sqoop schema to avro schema
      public AvroIntermediateDataFormat(org.apache.sqoop.schema.Schema schema) {
        super.setSchema(schema);
      } 
  • Implement a method to convert csv text to avro GenericRecord
    • private GenericRecord toAvro(String csv) {..}

  • Implement a method to convert the object array to avro GenericRecord
    • private GenericRecord toAvro(Object[] toObject(GenericRecord  data) { ..}

  • Conversely, implement a method to lazily construct the csv from avro GenericRecord when invoked
    • private String toCSV(GenericRecord record) { ..}

  • implement a method to lazily construct the object arrat from avro GenericRecord when invoked
    • private Object[] toObject(GenericRecord data) {...}

  • Implement methods to ser/ deser the avro record into a string - wire format

    Code Block
    /**
       * {@inheritDoc}
       */
      @Override
      public void write(DataOutput out) throws IOException {
       // todo
      }
      /**
       * {@inheritDoc}
       */
      @Override
      public void read(DataInput in) throws IOException {
        // todo
      }
    


  • Mappings from sqoop to avro types.

...