Status

Current state: Under DiscussionAccepted

Discussion thread: here [Change the link from the KIP proposal email archive to your own email thread]

JIRA: here [Change the link from KAFKA-1 to your own ticket]13595

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

...

ConsoleProducer will be updated to accept a new property: parsenull.value. It will default to "true" marker. By default, it will not be set to keep the existing behavior. When set to "false", the record value will be set to null.
Example, if any of the fields, key, value or headers (including individual header values, but not header keys) in the input match the marker, a record with that field set to null will be produced. The

Example:

$ ./bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic test --property null.marker=NULL
>NULL               (1)
>NULL more content  (2)

1. This will result in a record with value=null.
2. The record will have its value set to "NULL more content". The value has to exactly match the marker to be replaced by null.
   

Example with key and headers:

$ ./bin/kafka-console-producer.sh --bootstrap-server localhost:290929092 --topic test --property null.marker=NULL --property parse.valuekey=falsetrue --property parse.key=true
>mykey<tab>

...

headers=true
>h0:v0,h1:v1    NULL    myvalue    (1)
>NULL    mykey    NULL             (2)
>h0:,h1:v1    NULL    NULL         (3)
>h0:NULL,h1:v1    mykey    NULL    (4)
>NULL:v0    mykey    myvalue       (5)

...

1. This will result in a record with key=null, value="myvalue" and headers={"h0": "v0", "h1": "v1"}

1. This will result in a record with key="mykey", value=null and headers=null
2. This will result in a record with key=null, value=null and headers={"h0": "", "h1": "v1"}
3. This will result in a record with key="mykey", value=null and headers={"h0": null, "h1": "v1"}
4. This will return an error because headers cannot have a null key.

In case the specified null.marker value collides with any of the other flags, an error message will be returned to the user.

Proposed Changes

Describe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgement based on the scope of the changeThis will require small changes in ConsoleProducer/LineMessageReader to accept the new flag and update the generated ProducerRecord accordingly.

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users?
• If we are changing behavior how will we phase out the older behavior?
• If we need special migration tools, describe them here.
• When will we remove the existing behavior?

Rejected Alternatives

This is backwards compatible as the new flag will not be set by default.

Rejected Alternatives

NoneIf there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.