Status

Current state: Under DiscussionAccepted

Discussion thread: here

JIRA:

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Unlike for key-value state stores, Kafka Streams currently does not provide a way to query the range of keys available in a windowed state store.

...

Similarly, session stores currently only offer querying by a single given key, requiring the user to maintain the set of queryable keys separately. 

Public Interfaces

This KIP would add the following methods:

ReadOnlyWindowStore interface

WindowStoreIterator<KeyValue<KKeyValueIterator<Windowed<K>, V>> fetch(K from, K to, long timeFrom, long timeTo)

...

Key range behavior would be consistent with the existing ReadOnlyKeyValueStore.range(K from, K to) behavior.

SessionStore interface

KeyValueIterator<Windowed<K>, AGG> findSessions(final K keyFrom, final K keyTo, long earliestSessionEndTime, final long latestSessionStartTime);

Key range behavior would be consistent with the existing ReadOnlyKeyValueStore.range(K from, K to) behavior.

...

• Users implementing their own state stores would be affected by the interface changes.

Rejected Alternatives

Arguably, the The existing return type in ReadOnlyWindowStore.fetch((K key, long timeFrom, long timeTo) is not ideal, since WindowStoreIterator is already a KeyValueIterator KeyValueIterator<Long, V>, which abuses the key as a timestamp and the value as the object of interest. However , given that the existing API already abuses the iterator in a similar way, and barring more invasive changes to the API, or the introduction of type that embeds all of timestamp, key, and value – which would probably be a much larger discussion in itself – we want to be able to return the keys as part of range scans, so we considered using WindowStoreIterator<KeyValue<K, V>> as the return type for windowed stores for consistency between single key and range scan methods.

It was pointed out that doing so would limit the usefulness of peekNextKey() on the iterator, while also being somewhat confusing, since the iterator key doesn't actually contain the keys. As a result, it seemed simpler to follow the existing model.  model already used in ReadOnlySessionStore to return use a KeyValueIterator with a Windowed<K> as the key.