Status

Current state: [One of  "Under Discussion", "Accepted", "Rejected"]

Discussion thread: link

JIRA: Unable to render Jira issues macro, execution error.

Released: 0.4.0

Motivation

Finalize the stream operation primitives in distributedlog.

Public Interfaces

All supported stream primitives are listed as below.

PrimitiveStatusJiraNotes
#openLogDone Open a distributedlog stream
#deleteLogDone Delete a distributedlog stream
#getLogsDone List all the log streams under a given namespace
#markEndOfStreamDone Seal a log stream
#truncateDone Truncate a log stream
#getLogStatus Unable to render Jira issues macro, execution error.  Get the status/attributes of a log stream
#renameLog Unable to render Jira issues macro, execution error.  Rename a distributedlog stream
#listLogs Unable to render Jira issues macro, execution error.  List all the log streams under a given path
#symlink Unable to render Jira issues macro, execution error.  Create a symlink stream for a log stream
#fork Unable to render Jira issues macro, execution error.  Fork a log stream

 

Proposed Changes

GetLogStatus

Proposed API:

Future<LogStatus> getLogStatus(String streamName);

Proposed Changes:
Struct LogStatus {
	required long creation_time;
	required long modification_time;
	required int ensemble_size;
	required int write_quorum_size;
	required int ack_quorum_size;
	optional String symlinkLog;
	optional Map<byte[], byte[]> attributes;
}

 

The LogStatus is serialized and stored under the root znode of the log stream.

Rename

Proposed API:

Future<LogStatus> renameLog(String oldStreamName, String newStreamName);

Proposed Changes:

It will be a zookeeper transaction to copy all the znodes to the new location.

List

Proposed API:

Future<List<LogStatus>> listLogs(String logPath);

Proposed Changes:

It will get all the children under a given path and retrieve the `LogStatus` for each child stream.

Proposed API:

Future<Void> symlink(String realLogPath, String symlinkLogPath);

Proposed Changes:

It will create a log stream with LogStatus (symlink = 'real log path').

Fork

Proposed API: (Added to AsyncLogWriter)

Future<AsyncLogWriter> fork(String newLogStreamName);

 

AsyncLogWriter writer = ...;
AsyncLogWriter forkedWriter = FutureUtils.result(writer.fork('newStream'));
 

 

Proposed Changes:

At the forking time, the writer will do following:

  • Create a new log stream 'newStream'. The new log stream will copy the list of log segments, whose log segments will be symlinked to the original log segments.
  • Once the metadata is ready, the current writer will close the current in progress log segment.
  • All the writes happened after 'fork' will go to a new log segment.

Compatibility, Deprecation, and Migration Plan

 

N/A - Doesn't change existing primitives

Test Plan

N/A

Rejected Alternatives

N/A

  • No labels