THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
| Wiki Markup |
|---|
--DDL CREATE TABLE T ( pk STRING PRIMARY KEY NOT ENFOCED, sum_field1 BIGINT, max_field1 BIGINT, default_field BIGINT ) WITH ( 'merge-engine' = 'aggregation', 'aggregate-function' = '\{ sum_field1:sum, max_field2:max \}' -- sum up all sum_field1 with same pk; get max value of all max_field1 with same pk ); -- DML INSERT INTO T VALUES ('pk1', 1, 1, 1); INSERT INTO T VALUES ('pk1', 1, 1, NULL); -- verify SELECT * FROM T; => output 'pk1', 2, 2, NULL |
Tips: Columns which do not have designated aggregate functions using newest value to overwrite old value.
Future Work
An advanced way of introducing pre-aggregated merge into Flink table store is using materialized view to get pre-aggregated merge result from a source table. Then a stream job is started to synchronize data, consume source data, and write incrementally . This data synchronization job has no state. More information is described in JIRA.
...