Page History

The following is a draft design that uses a high-available consumer coordinator at the broker side to handle consumer rebalance. By migrating the rebalance logic from the consumer to the coordinator we can resolve the consumer split brain problem and help thinner the consumer client. With this design we hope to incorporates:

...

                             (record metadata)       -- (confirm) - -> Consumer (register succeed, block reading for coordinator requests such as ping, stop-fetcher, start-fetcher)

Coordinator       (record metadata)       - - (confirm) - -> Consumer (same as above)

...

                                     -- (ping) -->     Consumer (failed)

----------------------------------------------------------------------------------------------

...

                                     -- (stop-fetcher) - ->     Consumer (alive)

----------------------------------------------------------------------------------------------

...

                                                                                       - - (start-fetcher) - ->     Consumer (become failed)

...