Status
Current state: Under Discussion
Discussion thread: https://sematext.com/opensee/m/Kafka/uyzND1VU1Ou1y0Lbh?subj=+DISCUSS+KIP+466+Add+support+for+List+lt+T+gt+serialization+and+deserialization
JIRA:
Motivation
I believe there are many use cases where List Serde could be used. Ex. https://stackoverflow.com/questions/41427174/aggregate-java-objects-in-a-list-with-kafka-streams-dsl-windows, https://stackoverflow.com/questions/46365884/issue-with-arraylist-serde-in-kafka-streams-api
For instance, aggregate grouped (by key) values together in a list to do other subsequent operations on the collection.
Public Interfaces
- New class
org.apache.kafka.common.serialization.ListSerializer
which implements theSerializer<List<T>>
interface - New class
org.apache.kafka.common.serialization.ListDeserializer
which implementsDeserializer<List<T>>
interface - New subclass
ListSerde
inorg.apache.kafka.common.serialization.Serde
s which creates new serde based onListSerializer
andListDeserializer
classes
P.S. Static method corresponding to ListSerde under org.apache.kafka.common.serialization.Serdes class cannot be added because type needs to be defined beforehand. Something like this is not allowed:
Proposed Changes
This KIP proposes adding new ListSerializer and ListDeserializer classes as well as support for the new classes into the Serdes class. This will allow using List<T> Serde directly from Consumers, Producers and Streams.
List<T> serialization and deserialization will be done through repeatedly calling a serializer/deserializer for each entry provided by passed generic T's Serde. For example, if you want to create List of Strings serde, you will have to declare new ListSerde<>(Serdes.String()), in this case serializer/deserializer of String Serde will be used to serialize/deserialize each entry in `List<String>`.
Compatibility, Deprecation, and Migration Plan
Does not apply
Rejected Alternatives
Not known