...
ID | IEP-59 | ||||||||
Author | |||||||||
Sponsor | Nikolay IzhikovAnton Vinogradov | ||||||||
Created | 14.10.2020 | ||||||||
Status |
|
Table of Contents |
---|
Many use-cases build on observation and processing changed records.
...
draw.io Diagram | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
IgniteCDC Ignite CDC is a new utility that should be run on the server node host. CDC utility watches by the appearance of the WAL archive segments.
On the segment archiving, the utility iterates it using the existing WALIterator WAL Iterator and notifications CDCConsumer CDC Consumer of each record from the segment.
CDCConsumer public Public API interface:
Code Block | ||||
---|---|---|---|---|
| ||||
/** Consumer of data change events. */ @IgniteExperimental public interface CaptureDataChangeConsumerChangeDataCaptureConsumer { /** * Starts the consumer. */ * @param log Logger. */ public void start(IgniteLogger log); /** * Handles entry changes events. * If this method return {@code true} then current offset will be stored * and ongoing notifications after CDC application fail/restart * will be started from it. * * @param events Entry change events. * @return {@code True} if current offset should be saved on the disk * to continue from it in case any failures or restart. */ public boolean onChangeonEvents(Iterator<ChangeEvent>Iterator<ChangeDataCaptureEvent> events); /** * Stops the consumer. * This methods can be invoked only after {@link #start(IgniteLogger)}. */ public void stop(); } /** * Event of single entry change. * Instance presents new value of modified entry. * * @see IgniteCDC * @see CaptureDataChangeConsumer */ @IgniteExperimental public interface ChangeEventChangeDataCaptureEvent extends Serializable { /** * @return Key for the changed entry. */ public Object key(); /** * @return Value for the changed entry or {@code null} in case of entry removal. */ @Nullable public Object value(); /** * @return {@code True} if event fired on primary node for partition containing this entry. * @see <a href=" * https://ignite.apache.org/docs/latest/configuring-caches/configuring-backups#configuring-partition-backups">Configuring> * Configuring partition backups.</a> */ public boolean primary(); /** * Ignite split dataset into smaller chunks to distribute them across the cluster. * {@link ChangeDataCaptureConsumer} implementations can use {@link #partition()} to split changes processing * in the same way as it done for the cache. * * @return Partition number. */ @see Affinity#partition(Object) public* int@see partitionAffinity#partitions(); /** @see <a href="https://ignite.apache.org/docs/latest/data-modeling/data-partitioning">Data partitioning</a> * @return Order of* the@see update operation.<a href="https://ignite.apache.org/docs/latest/data-modeling/affinity-collocation">Affinity collocation</a> */ public ChangeEventOrderint orderpartition(); /** * @return Version of Operationthe typeentry. */ public ChangeEventTypeCacheEntryVersion operationversion(); /** * @return Cache ID. * @see org.apache.ignite.internal.util.typedef.internal.CU#cacheId(String) * @see CacheView#cacheId() */ public int cacheId(); } /** * Entry event order. * Two concurrent updates of the same entry can be ordered based on {@link ChangeEventOrder} comparsion. * Greater value means that event occurs later. */ @IgniteExperimental public interface ChangeEventOrderCacheEntryVersion extends Comparable<ChangeEventOrder>Comparable<CacheEntryVersion>, Serializable { /** @return topVer Topology version plus* numberOrder of seconds from the start time the update. Value is an incremental counter value. Scope of thecounter firstis grid node. * @return Version order. */ public intlong topVerorder(); /** @return nodeOrderDrId Node order and DR ID on which this version was assigned. */ public int nodeOrderDrIdnodeOrder(); /** @return Data* center id. */ Cluster id is a value to distinguish updates in case user wants to aggregate and sort updates from several * Ignite clusters. {@code clusterId} id can be set for the node using public* byte{@link dataCenterIdGridCacheVersionManager#dataCenterId(byte); }. * /* * @return order Version order. Cluster id. */ public longbyte orderclusterId(); /** @return ReplicationTopology version plus number of seconds from the start time of the first grid node. */ public ChangeEventOrderint otherDcOrdertopologyVersion(); } /** * If Typesource of operations the update is "local" cluster then {@code null} will be returned. * * @see IgniteCDC * @seeIf CaptureDataChangeConsumer updated *comes @seefrom ChangeEvent */ @IgniteExperimental public enum ChangeEventType {the other cluster using {@link IgniteInternalCache#putAllConflict(Map)} /** Update operation. */ UPDATE, * then entry version for other cluster. * @return Replication version. * @see IgniteInternalCache#putAllConflict(Map) /** Delete operation.* @see IgniteInternalCache#removeAllConflict(Map) */ DELETEpublic CacheEntryVersion otherClusterVersion(); } |
Risks and Assumptions
https://dev.mysql.com/doc/refman/8.0/en/mysqlbinlog.html
...
Jira | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|