...

Current state: [ UNDER DISCUSSION | ACCEPTED | REJECTED  ]

Discussion thread: <link to mailing list DISCUSS thread>

...

/**
 * An AdjunctDataStoreManager instance is responsible for
 * 1. maintaining the mapping between system streams and adjunct data stores
 * 2. populating adjunct data stores
 */
public interface AdjunctDataStoreManager {
   /**
    * Invoked before a message is passed to a task
    * @returns true if the message is saved in a store, false otherwise
    */
    AdjunctDataStoreManager boolean process(IncomingMessageEnvelop message);
}

/**
 * A factory to instantiate converters
 */
public interface ConverterFactory {
    Converter getConvert(Config config);
}

/**
 * A converter converts an input object to another type
 */
public interface Converter {
   /**
    * Converts an input object to another type
    *
    * @param input the object to be converted
    * @returns the converted object
    */
    Object convert(Object input);
}

No changes to public interface

Implementation and Test Plan

• Introduce interfaces and implementation described above
• Add unit tests to test and verify functionality (plain, bootstrap, broadcast, ...)
• Verify in different deployment environment, note: implementation should be agnostic to deployment environment (YARN, Hadoop or standalone). 

Compatibility, Deprecation, and Migration Plan

 As this is a new feature, no plans are required for compatibility, deprecation and migration.

Rejected Alternatives

...