Status
...
Page properties | |
---|---|
|
...
JIRA: TBD
...
|
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
This FLIP is a joint work of initially proposed by Yuan Zhu (zstraw@163.com), and finished by Qingsheng Ren (partial caching part) and Alexander Smirnov (full caching part).
Table of Contents |
---|
Motivation
...
Code Block | ||||
---|---|---|---|---|
| ||||
/** * A semi-persistent mapping from keys to values for storing entries of lookup table. * * <p>The type of the caching key is a {@link RowData} with lookup key fields packed inside. The * type of value is a {@link Collection} of {@link RowData}, which are rows matching lookup key * fields. * * <p>Cache entries are manually added using {@link #put}, and are stored in the cache until either * evicted or manually invalidated. * * <p>Implementations of this interface are expected to be thread-safe, and can be safely accessed * by multiple concurrent threads. */ @PublicEvolving public interface LookupCache extends AutoClosable, Serializable { /** * Initialize the cache. * * @param metricGroup the metric group to register cache related metrics. */ void open(CacheMetricGroup metricGroup); /** * Returns the value associated with key in this cache, or null if there is no cached value for * key. */ @Nullable Collection<RowData> getIfPresent(RowData key); /** * Associates the specified value rows with the specified key row in the cache. If the cache * previously contained value associated with the key, the old value is replaced by the * specified value. * * @return the previous value rows associated with key, or null if there was no mapping for key. * @param key - key row with which the specified value is to be associated * @param value – value rows to be associated with the specified key */ Collection<RowData> put(RowData key, Collection<RowData> value); /** Discards any cached value for the specified key. */ void invalidate(RowData key); /** Returns the number of key-value mappings in the cache. */ long size(); } |
...
Option | Type | Descriptions | |
---|---|---|---|
lookup.cache | Enum of NONE, PARTIAL and FULL | The caching strategy for this lookup table. NONE: Do not use cache Partial: Use partial caching mode FULL: Use full caching mode | |
lookup.max-retries | Integer | The maximum allowed retries if a lookup operation fails | |
lookup.partial-cache.expire-after-access | Duration | Duration to expire an entry in the cache after accessing | |
lookup.partial-cache.expire-after-write | Duration | Duration to expire an entry in the cache after writing | |
lookup.partial-cache.cache-missing-key | Boolean | Whether to store an empty value into the cache if the lookup key doesn't match any rows in the table | |
lookup.partial-cache.max-rows | Long | The maximum number of rows to store in the cache | |
lookup.full-cache.reload-strategy | Enum of PERIODIC and TIMED | The reload strategy for the full cache scenario. PERIODIC: Use PeriodicCacheReloadTrigger TIMED: Use TimedCacheReloadTrigger | |
lookup.full-cache.periodic-reload.interval | Duration | Duration to trigger reload in the PeriodicCacheReloadTrigger | |
lookup.full-cache.periodic-reload.schedule-mode | Enum of FIXED_DELAY and FIXED_RATE | The periodically schedule mode of reloading in the PeriodicCacheReloadTrigger | |
lookup.full-cache.timed-reload-at.iso-time | LocalTime | String | Time in ISO-8601 format when cache needs to be reloaded. Time can be specified either with timezone or without timezone (target JVM local timezone will be used). See formatter ISO_TIME. The local time on the target JVM to trigger a reload |
lookup.full-cache.timed-reload.interval-in-days | Integer | The interval in days to trigger the reload at the specified time |
...