Status

Current state: Done.

Release: 4.5.0

Problem

Using BookKeeper as a low-latency distributed data storage achiving read-your-writes consistency outside the LAC protocol

The primary purpose of BookKeeper is to be used as distributed transaction log, but for its internal design it can be used as a distributed storage of binary data.

...

The Idea is to add a new BookKeeper client side option side  method "allowUnsafeReadsreadUnconfirmedEntries" to skip the check

if (lastEntry > lastAddConfirmed)

(and the async counterpart) which does not check for the LastAddConfirmed range

public void asyncReadUnconfirmedEntries(long firstEntry, long lastEntry, ReadCallback cb, Object ctx) {
        // Little sanity check
        if (firstEntry < 0 || firstEntry > lastEntry) {
            LOG.error("IncorrectParameterException on ledgerId:{} firstEntry:{} lastEntry:{}",
                    new Object[] { ledgerId, firstEntry, lastEntry });
            cb.readComplete(BKException.Code.IncorrectParameterException, this, null, ctx);
            return;
        }
        asyncReadEntriesInternal(firstEntry, lastEntry, cb, ctx);
    }

 This way the reader will be able to read data from Bookies, assuming its own responsability on the consistency of data.

We can say that there is an out-of-band contract between the writer and the readers, we only want to achieve "read-your-writes consistency"

Actions

• Related JIRA:

  Jira
  server ASF JIRA serverId

Actions

...

• 5aa69414-a9e9-3523-82ec-879b028fb15b key BOOKKEEPER-1019

• Related PR: https://github.com/apache/bookkeeper/pull/121

Rejected Alternatives:

• Add a ClientConfiguration option 'readEntriesAllowReadAfterLastAddConfirmed' to disable the check for the method readEntries.
  • This has been rejected (Sijie Guo) because the usage of a new method will be clearer and because if a client is shared among different reader procedure this option will introduce uncontrolled side-effects

Related works:

• In Yahoo private fork the a similar work has been done in the BookKeeperAdmin class (see https://github.com/yahoo/bookkeeper/commit/92e2f906f83d997b5cf5ffd2bcfcdbe53ca3bd04) (Unknown User (mmerli))
• In Salesforce private fork the 'ExplicitLAC' feature (contributed to 4.5.0 Apache version) is used to decrease overall latency of writes, without violating the LAC protocol (Unknown User (jujjuri))

...