This page is meant as a template for writing a DP (DistributedLog Proposal). To create a DP choose DP-Template on creating a page and modify with your content and replace the heading with the next DP number and a description of your issue. Replace anything in italics with your own description.

Status

Current state: [One of  "Under Discussion", "Accepted", "Rejected"]

Discussion thread: [link]

JIRA:  [link] DL-63

Released: <DL version>

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Describe the problems you are trying to solve.

Public Interfaces

Briefly list any new interfaces that will be introduced as part of this proposal or any existing interfaces that will be removed or changed. The purpose of this section is to concisely call out the public contract that will come along with this feature.

A public interface is any change to the following:

• Data format, Metadata format

• The wire protocol and api behavior

• Any class in the public packages

• Monitoring

• Command line tools and arguments

• Anything else that will likely break existing users in some way when they upgrade

Proposed Changes

Describe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgement based on the scope of the change.

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users? 
• If we are changing behavior how will we phase out the older behavior? 
• If we need special migration tools, describe them here.
• When will we remove the existing behavior?

Test Plan

Describe in few sentences how the DP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

Rejected Alternatives

0.4.0

Motivation

The motivation is discussed at http://mail-archives.apache.org/mod_mbox/incubator-distributedlog-dev/201610.mbox/browser

Public Interfaces

Epoch Write - is also called 'session write'. There will be two new methods added to the DistributedLogClientBuilder for building a DistributedLogClient.

These two new build methods are:

1. enableSessionWrite(boolean) - The build method is to enable session write for the client built by this builder. By default, it will be false.
2. enableExclusiveSession(boolean) - The build method is to enable exclusive session for the client built by this builder. By default, it will be false. This flag only take effects when #enableSessionWrite is set to true.

There will be one new method added to DistributedLogClient for supporting fencing session operation discussed in http://mail-archives.apache.org/mod_mbox/incubator-distributedlog-dev/201610.mbox/browser

/**
 * Retrieve the last DLSN of stream <i>streamName</i>.
 * <p>The operation will fence the session if <i>fenceSession</i> is true and session write is enabled (via #enableSessionWrite(true)).
 * The DLSN returned will be the last DLSN of the stream in current session, no records can be written by the clients that hold the old
 * session.
 *
 * @param streamName the name of the stream
 * @param fenceSession the flag to fence the session. it takes no effects if session write is disabled.
 */
Future<DLSN> getLastDLSN(String streamName, boolean fenceSession); 

Example to build the client is described as below:

// Build the client with session write enabled.
 
DistributedLogClient client = DistributedLogClientBuilder.newBuilder()
                                 ...
                                 .enableSessionWrite(true)
                                 .build();
 
// Example to use the session future.
 
String streamName = ...;
ByteBuffer record = ...;


client.write(streamName, record) onSuccess {
 // write the next record
} onFailure {
 // we don't know if the record is written successfully or not
 // get the last DLSN and fence the current session
 Future<DLSN> lastDLSN = client.getLastDLSN(streamName, true /*fence the session */);
 lastDLSN map { dlsn =>
 // read the records from last known dlsn and the current dlsn to check duplications.
 }
}
 

Proposed Changes

A 'session' is an identify the lifespan of the ownership of log stream. It is a positive long number - It will be bumped when the ownership of the log stream is changed or explicitly bumped by getLastDLSN(stream, true) call. We will basically use the `transaction id` of log records in the log stream for assigning sessions.

When the log stream is acquired by write proxy (ownership changed) or the session is explicitly bumped, the owner of the log stream will first advance the transaction id generator to claim a new transaction id and write a control record to the log stream. Upon successfully written the control record, the owner of the log stream successfully bump the session id. The transaction id of this control record will be used as current session id until the ownership is changed again or session is explicitly bumped.

Wire Protocol Changes

We need to make following changes to the write protocol.

// Status Code
SESSION_FENCED
 
// HeartBeat Options
struct HeartbeatOptions {
    1: optional bool sendHeartBeatToReader;
    2: optional bool getSession; /** NEW FIELD: retrieve current session of the stream **/
    3: optional bool exclusiveSession; /** NEW FIELD: shall write proxy promote the session to be an exclusive session **/
}
 
// Write Response
struct WriteResponse {
    1: required ResponseHeader header;
    2: optional string dlsn;
    3: optional i64 session; /** NEW FIELD: return the current session of the stream **/
}
 
// Write Context
struct WriteContext {
    1: optional set<string> triedHosts;
    2: optional i64 crc32;
    3: optional bool isRecordSet;
	4: optional i64 session; /** NEW FIELD: carry the current session that the client knows **/
}
 

Write Proxy Changes

• Handle write request - it will check if the write request carries a session id. if there is a session id, it would compare its current session id with the session id carried by the request. Then it would accept write request if the session ids are matched; otherwise, the write proxy will respond SESSION_FENCED.
• New RPC - 'getLastDLSN' :
  • If `fenceSession` is set to true, it will write a control record to bump the session id and upon successfully written the control record, it will respond the DLSN of this control record as the response for the rpc.
  • If `fenceSession` is set to false, it will just return the DLSN of last acknowledged record as the response for this rpc.

Client Changes

• On finding the ownership of the stream
  • Change the client to use heartbeat rather than the normal write to find the ownership of the stream.
  • The heartbeat options will be instantiated based on the builder (set `getSession` to true if #enableSessionWrite is enabled, set `exclusiveSession` to true if #enableExclusiveSession is enabled)
• On successfully finding the ownership of the stream, it will receive the session of the stream in the WriteResponse of heartbeat request.
• All the subsequent write request will use this session id to construct their WriteContext.
• Upon receiving SESSION_FENCED response, the writes will be failed. However the client will not automatically retrieve the new session.   

Compatibility, Deprecation, and Migration Plan

All the features are off by default. So there is no backward compatibility concern.

No methods are deprecated.

Upgrade Plan

As we are adding new status code, we need to first update the write proxy to make sure all write proxies know the new protocol, then update the clients.

Test Plan

The test plan will cover following cases:

1. A client (enabled session write) talks to a proxy server (enabled session write)
2. A client (enabled session write) talks to an old proxy server (no session write support)
3. An old client (no session write) talks to a proxy server (enabled session write)
4. Mixed usages
   1. A client (enabled session write) with an old client (no session write)
   2. A client (enabled session write and also exclusive session) with A client (enabled session write)
   3. A client (enabled session write and also exclusive session) with A client (disabled session write)

Rejected Alternatives

N/AIf there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.