Page History

...

The following structures are used for the initial handshake between client and server:

    class LearnerInfo {
        int protocolVersion;
        int serverid;
    }

The actual proposals are communicated between follower and leader using a single data structure:

class QuorumPacket {
    int type; // Request, Ack, Commit, Ping, etc
    long zxid;
    buffer data;
    vector<org.apache.zookeeper.data.Id> authinfo; // only used for requests
}

Here are the types of the QuorumPackets:

Zab servers have the following state:

Implementation assumptions

...

1. l The leader starts accepting connections from followers.
2. f Followers connect the the leader and send FollowerInfoFOLLOWERINFO.
3. l Once the leader has quorum, it stops accepting connections, and sends ServerInfo LEADERINFO(e) to all followers, where e is greater than all f.acceptedEpoch in the quorum.
4. f When the follower receives ServerInfo LEADERINFO(e) it will do one of the following, where e = ServerInfo.proposedEpoch:
   • if e > f.acceptedEpoch, the the follower sets f.acceptedEpoch = e and sends ACKACKEPOCH(e<<32e);
   • if e == f.acceptedEpoch, the follower does not send AckNewEpochACKEPOCH, but continues to next step;
   • if e < f.acceptedEpoch, the follower closes the connection to the leader and goes back to leader election;
5. l The leader waits for a quorum of followers to send AckNewEpochACKEPOCH.
6. l If the following conditions are not met for all connected followers, the leader disconnects followers and goes back to leader election:
   • f.currentEpoch <= l.currentEpoch
   • if f.currentEpoch == l.currentEpoch, then f.lastZxid <= l.lastZxid

...

There are two noticeable differences between this description of phase 1 and the one found in http://research.yahoo.com/files/YL-2010-007.pdf:DISSECTING ZAB

1. the leader does not sync with the most up-to-date follower
2. the followers do not send their histories in the ACK of the NEWEPOCH message.

...

Another difference from the protocol described in the tech report is the extra information sent in ServerInfo LearnerInfo and the FollowerInfo. The extra information is for implementation purposes: making sure that the protocol version is compatible between client and server, and information for debugging purposes.

...

1. l The leader does the following with each follower connected to it:
   1. adds the follower to the list of connections to send new proposals, so while the server is performing the next steps, it is queuing up any new proposals sent to the follower.
   2. does one of the following:
      • SNAP if the follower is so far behind that it is better to do a state transfer than send missing transactions.
      • TRUNC(zxid) if the follower has transactions that the leader has chosen to skip. The leader sets zxid to the last zxid in its history for the epoch of the follower. The leader then sends the transactions that the follower is missing.
      • DIFF if the leader is sending transactions that the follower is missing. The leader sends missing messages to the follower.
   3. sends a NEWLEADER(e);
   4. The leader releases any queued messages to the follower.
2. f The follower syncs with the leader, but doesn't modify its state until it receives the NEWLEADER(e) packet. Once it receives NEWLEADER(e) it atomically applies the new state and sets f.currentEpoch = e. It then sends ACK(e << 32).
3. l Once the leader has received an acknowledgements from a quorum of followers, it takes leadership of epoch e and queues UPTODATE to all followers.
4. f When a follower receives the UPDATE UPTODATE message, it starts accepting client connections and serving new state.
5. l Leader starts accepting connections from followers again. The variable nextZxid is set to (e << 32) + 1.

...