Key

Default

Type

Description

table-storage.log.kafka.bootstrap.servers

(none)

MapString

Kafka brokers. eg: localhost:9092

table-storage.log.retention

(none)

Duration

It means how long changes log will be kept. The default value is from the log system cluster.

Specifies the scan startup log consistency mode for log consumertable.

• full: Performs a snapshot on the table upon first startup, and continue to read the latest changes. (Using HybridSource, the switching between snapshot and changes is exactly-once consistency because we store the offset of the corresponding log to snapshot when writing data)
• latest: Start from the latest.
• from-timestamp: Start from user-supplied timestamp.

Specifies the log consistency mode for table.

• transactional: only the data after the checkpoint transactional: only the data after the checkpoint can be seen by readers, the latency depends on checkpoint interval
• eventual: Immediate data visibility, you may see some intermediate states, but eventually the right results will be produced, only works in table with primary key

Specifies the log changelog mode for table with primary key.

• auto: upsert for table with primary key, all for table without primary key.
• upsert: the log system does not store the UPDATE_BEFORE changes, the log consumed job will automatically add the normalized node, relying on the state to generate the required update_before.
• all: the log system stores all changes including UPDATE_BEFORE

table-storage.file.path

(none)

String

Root file path.

table-storage.file.format

parquet

String

Format name for file.

table-storage.bucket

1

Integer

Bucket number for file and Partition number for Kafka.