Status
...
Page properties | |
---|---|
|
...
...
JIRA: <TODO>
...
|
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
name | type | description |
name | String | catalog.database.tableName |
log.system | String | the log system |
log.kafka.bootstrap.servers | Map | Kafka brokers |
log.retention | Duration | how long changes log will be kept |
file.path | String | File path |
log.kafka.topic | String | topic of Kafka |
file.format | String | format for file |
bucket | Integer | bucket number |
change-tracking | Boolean | does this table tracking changes |
...
Key | Default | Type | Description |
table-storage.log.system | kafka | String | Log system. Now only Kafka in the MVP. |
table-storage.log.kafka.bootstrap.servers | (none) | MapString | Kafka brokers. eg: localhost:9092 |
table-storage.log.retention | (none) | Duration | It means how long changes log will be kept. The default value is from the log system cluster. |
table-storage.log.consistency | transactional | String | Specifies the log consistency mode for table.
|
table-storage.log.changelog-mode | auto | String | Specifies the log changelog mode for table.
|
table-storage.log.key.format | json | String | Specifies the key message format of log system with primary key. |
table-storage.log.format | debezium-json | String | Specifies the message format of log system. |
table-storage.file.path | (none) | String | Root file path. |
table-storage.file.format | parquet | String | Format name for file. |
table-storage.bucket | 1 | Integer | Bucket number for file and Partition number for Kafka. |
...
In addition to session options that can be configured individually for each table by removing the prefix, there are also some options that can be configured individually only for tables, they are the options that affect reading and writing:
Key | Default | Type | Description |
log.scan | full | String | Specifies the scan startup mode for log consumer.
|
log.scan.timestamp-mills | (none) | Long | Optional timestamp used in case of "from-timestamp" scan mode. |
change-tracking | true | Boolean | If users do not need to consume changes from the table, they can disable Change Tracking. This can reduce resource consumption. |
Bucket
The record is hashed into different buckets according to the primary key (if have) or the whole row (without primary key):
...