Page History

Overview

This guide describes how to implement custom data streamer and describes basic concepts and implementation details.

...

• conversion of stream specific messages to Java object (e.g. byte array to String);
• stream wrapper that encapsulates functionality related with particular data source.

Data From connection initiating stand point data streamer can be implemented as a server or a client with respect to the of data source and requirmentsrequirements. For example, HTTP data streamer that implemented as a client can request the data from external web services. On the other hand, HTTP data streamer that implemented as a server can process a large number of request from internal micro servicesexternal HTTP clients.

In order to implement custom data streamer a developer should optionally extend StreamAdapter class, add functionality related with particular data source and streamer life cycle logic if needed. As mentioned above StreamAdapter wraps IgniteDataStreamer instance and also needs StreamTupleExtractor instance, that could be provided via constructor or corresponding setters.

...

• message size based protocol (default) where each message in the stream is prepended by 4-byte integer header containing message size;
• message delimiter based protocol where each message in the stream is appended with user defined delimiter (see SocketStreamer.setDelimiter(byte[] delim) method).

Reference Implementation

Streamer Life Cycle

Streamer life cycle depends on chosen implementation model (server or client) and requirements. Usually streamer can be in one of the following states: created, initialized, shutdown and may be one of transition states. StreamAdapter doesn't provide any life cycle managements methods. Correct life cycle management implementation is completely responsibility of data streamer developer.

See SocketStreamer implementation with life cycle management start and stop methods that initialize and shutdown TCP socket server respectively.

Implementation Tips