Status
Current state: Under Discussion
Discussion thread: https://lists.apache.org/thread/lxfq8y8co76mjwxhowwx6cjl0hw1mxpy
JIRA:
Motivation
org.apache.kafka.connect.data.Schema
to describe record structure. There are two methods boolean isOptional()
and Object defaultValue()
may cause confused.
private JsonNode convertToJson(Schema schema, Object value) { if (value == null) { if (schema == null) return null; if (schema.defaultValue() != null) return convertToJson(schema, schema.defaultValue()); if (schema.isOptional()) return JsonNodeFactory.instance.nullNode(); throw new DataException("Conversion error: null value for field that is required and has no default value"); } ... }
isOptional()
return true
and defaultValue()
return not null
, in other word, an optional field which has default value, if we use org.apache.kafka.connect.json.JsonConverter
to serialize the field, will eventually get defaultValue()
create table t1 { name varchar(40) not null, create_time datetime default '1999-01-01 11:11:11' null, update_time datetime default '1999-01-01 11:11:11' null }
INSERT INTO `t1` (`name`, `update_time`) VALUES ('kafka', null);
the record will store in MySQL as:
{ "name": "kafka", "create_time": "1999-01-01 11:11:11", "update_time": null }
debezium
{ "name": "kafka", "create_time": "1999-01-01 11:11:11", "update_time": "1999-01-01 11:11:11" }
null
, we can treat it as null
or default value
Public Interfaces
Add a config property `serialize.accept.optional.null`, default `false`, for compatibility.
Proposed Changes
org.apache.kafka.connect.storage.ConverterConfig
, default false
ny third party converters can extend ConverterConfig
just like JsonConverterConfig to read this property and implement it.
Serialization(convertToJson), for an optional null field which has default value,
if set serialize.accept.optional.null=true, take null;
if set serialize.accept.optional.null=false, take default value.
Compatibility, Deprecation, and Migration Plan
The change will not break the compatibility.
Rejected Alternatives
null
on an optionalnull