Status

Current state: Under Discussion

Discussion thread: link

JIRA: DL-63

Released: 0.4.0

Motivation

The motivation is discussed at http://mail-archives.apache.org/mod_mbox/incubator-distributedlog-dev/201610.mbox/browser

Public Interfaces

Epoch Write - is also called 'session write'. There will be two new methods added to the DistributedLogClientBuilder for building a DistributedLogClient.

These two new build methods are:

  1. enableSessionWrite(boolean) - The build method is to enable session write for the client built by this builder. By default, it will be false.
  2. enableExclusiveSession(boolean) - The build method is to enable exclusive session for the client built by this builder. By default, it will be false. This flag only take effects when #enableSessionWrite is set to true.

There will be one new method added to DistributedLogClient for supporting fencing session operation discussed in http://mail-archives.apache.org/mod_mbox/incubator-distributedlog-dev/201610.mbox/browser

 


/**
 * Retrieve the last DLSN of stream <i>streamName</i>.
 * <p>The operation will fence the session if <i>fenceSession</i> is true and session write is enabled (via #enableSessionWrite(true)).
 * The DLSN returned will be the last DLSN of the stream in current session, no records can be written by the clients that hold the old
 * session.
 *
 * @param streamName the name of the stream
 * @param fenceSession the flag to fence the session. it takes no effects if session write is disabled.
 */
Future<DLSN> getLastDLSN(String streamName, boolean fenceSession); 


Example to build the client is described as below:

 

// Build the client with session write enabled.
 
DistributedLogClient client = DistributedLogClientBuilder.newBuilder()
                                 ...
                                 .enableSessionWrite(true)
                                 .build();
 
// Example to use the session future.
 
String streamName = ...;
ByteBuffer record = ...;


client.write(streamName, record) onSuccess {
 // write the next record
} onFailure {
 // we don't know if the record is written successfully or not
 // get the last DLSN and fence the current session
 Future<DLSN> lastDLSN = client.getLastDLSN(streamName, true /*fence the session */);
 lastDLSN map { dlsn =>
 // read the records from last known dlsn and the current dlsn to check duplications.
 }
}
 

Proposed Changes

A 'session' is an identify the lifespan of the ownership of log stream. It is a positive long number - It will be bumped when the ownership of the log stream is changed or explicitly bumped by getLastDLSN(stream, true) call. We will basically use the `transaction id` of log records in the log stream for assigning sessions.

When the log stream is acquired by write proxy (ownership changed) or the session is explicitly bumped, the owner of the log stream will first advance the transaction id generator to claim a new transaction id and write a control record to the log stream. Upon successfully written the control record, the owner of the log stream successfully bump the session id. The transaction id of this control record will be used as current session id until the ownership is changed again or session is explicitly bumped.

Wire Protocol Changes

We need to make following changes to the write protocol.

// Status Code
SESSION_FENCED
 
// HeartBeat Options
struct HeartbeatOptions {
    1: optional bool sendHeartBeatToReader;
    2: optional bool getSession; /** NEW FIELD: retrieve current session of the stream **/
    3: optional bool exclusiveSession; /** NEW FIELD: shall write proxy promote the session to be an exclusive session **/
}
 
// Write Response
struct WriteResponse {
    1: required ResponseHeader header;
    2: optional string dlsn;
    3: optional i64 session; /** NEW FIELD: return the current session of the stream **/
}
 
// Write Context
struct WriteContext {
    1: optional set<string> triedHosts;
    2: optional i64 crc32;
    3: optional bool isRecordSet;
	4: optional i64 session; /** NEW FIELD: carry the current session that the client knows **/
}
 

Write Proxy Changes

  • Handle write request - it will check if the write request carries a session id. if there is a session id, it would compare its current session id with the session id carried by the request. Then it would accept write request if the session ids are matched; otherwise, the write proxy will respond SESSION_FENCED.
  • New RPC - 'getLastDLSN' :
    • If `fenceSession` is set to true, it will write a control record to bump the session id and upon successfully written the control record, it will respond the DLSN of this control record as the response for the rpc.
    • If `fenceSession` is set to false, it will just return the DLSN of last acknowledged record as the response for this rpc.

Client Changes

  • On finding the ownership of the stream
    • Change the client to use heartbeat rather than the normal write to find the ownership of the stream.
    • The heartbeat options will be instantiated based on the builder (set `getSession` to true if #enableSessionWrite is enabled, set `exclusiveSession` to true if #enableExclusiveSession is enabled)
  • On successfully finding the ownership of the stream, it will receive the session of the stream in the WriteResponse of heartbeat request.
  • All the subsequent write request will use this session id to construct their WriteContext.
  • Upon receiving SESSION_FENCED response, the writes will be failed. However the client will not automatically retrieve the new session.   

Compatibility, Deprecation, and Migration Plan

All the features are off by default. So there is no backward compatibility concern.

No methods are deprecated.

Upgrade Plan

As we are adding new status code, we need to first update the write proxy to make sure all write proxies know the new protocol, then update the clients.

Test Plan

The test plan will cover following cases:

  1. A client (enabled session write) talks to a proxy server (enabled session write)
  2. A client (enabled session write) talks to an old proxy server (no session write support)
  3. An old client (no session write) talks to a proxy server (enabled session write)
  4. Mixed usages
    1. A client (enabled session write) with an old client (no session write)
    2. A client (enabled session write and also exclusive session) with A client (enabled session write)
    3. A client (enabled session write and also exclusive session) with A client (disabled session write)

Rejected Alternatives

N/A

  • No labels