Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Status

Current state["Under Discussion"]

...

Discussion thread

...

...

...

...

Released: <Flink Version>

...

kfqkmtlpk2k3q3cc3l8p0j7lg6b3o0sj
JIRA

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyFLINK-31487

Release<Flink Version>

Motivation

To support connectors in avoiding overwriting non-target columns with null values when processing partial column updates, we propose adding information on the target column list to DynamicTableSink#Context.

...

The current connector implementor has no way of knowing that the last three fields were added by the planner and not from real user data. The user has to declare several different schemas (containing only partial column information and no overlap except for the primary key) to get around the current problem. 

By adding targetColumnList target column list information to the DynamicTableSink#Context, this problem can be solved.

...

Add new getTargetColumns to DynamicTableSink#Context.

Code Block
languagejava
           /**
         * Returns an {@link Optional} array of column index paths related to user specified target
         * column list or an empty{@link Optional#empty()} when not specified. The array indices are 0-based
 and support
       * and *support composite columns within (possibly nested) structures.
         *
         * <p>This information comes from the column list of the DML clause, e.g., for a sink table
         * t1 which schema is: {@code a STRING, b ROW < b1 INT, b2 STRING>, c BIGINT}
         *
         * <ul>
         *   <li>insert: 'insert into t1(a, b.b2) ...', the column list will be 'a, b.b2', and will
         *       return {@code [[0], [1, 1]]}. The columnstatement list'insert willinto betarget empty for 'insert into targetselect ...' without
         *       specifying a  select ...'.column list will return {@link Optional#empty()}.
         *   <li>update: 'update target set a=1, b.b1=2 where ...', the column list will be 'a,
         *       b.b1', will return {@code [[0], [1, 0]]}.
         * </ul>
         *
         * <p>Note: will always return empty for the delete statement because it has no column list.
         */
            Optional<int[][]> getTargetColumns();

...