This guide shows how to implement custom data streamer and describes basic concepts and implementation details.
The purpose of streamers is data ingesting from various sources (e.g. TCP socket or Kafka) and streaming them into Ignite cache for further processing. The Ignite provides IgniteDataStreamer
API that allows stream data into cache and convenient StreamAdapter
that wraps data streamer instance and provides basis for custom streamer implementation.
Data stream consists of messages that can be represented as Java objects of some type that should be converted to cache entries (key-value pairs or tuples). StreamTupleExtractor
is responsible for such conversion.
The following parts of custom data streamer are missing and should be implemented by developer:
String
);Data streamer can be implemented as server or client with respect to the data source and requirments. For example HTTP data streamer that implemented as client can request the data from external web services. On the other hand, HTTP data streamer that implemented as server can process a large number of request from internal micro services.