Versions Compared

Old Version 1

changes.mady.by.user Sijie Guo

Saved on

compared with

New Version 2

changes.mady.by.user Sijie Guo

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Attendees:

Discuss

BP-1 - 64 bits ledger id 3 - Security support

NOTES

    • ACL

      • Leverage zookeeper metadata for ACL manage

      • It might be worth thinking of a common ACL for bookkeeper (both data and metadata) and map the bookkeeper ACL back to zookeeper ACL for the metadata part. (Sijie Guo)

    • SASL - Bookie & Auditor? Can we share the entry?

    • Authentication

      • Leverage the authentication framework in 4.4.0

      • However it is using a shaded protobuf for the public interface. There is a pull request to change it.

    • Rolling upgrade

      • Discussed the rolling restart sequence.

    • How to handle eertification expiry??

      • Background check the certification periodically

    • Rolling upgrade with certification

    • StartTLS

      • Plain communication

      • StartTLS

      • Enrico send points

      • Leverage Ivan Kelly's patch and Salesforce's patch

    • JV shares the work that Salesforce did in TLS/SSL part.

      • share the same connection

  • Background Introduction

    • the interface, wire protocol and storage all support 64 bits ledger id.

    • however, the ledger id generation on zookeeper only generate 32 bits ledger id.
    • the zookeeper-based ledger metadata store uses 2-4-4 split to store 32 bits ledger id.
    • we need to find a solution to support 64 bits in zookeeper-based ledger metadata store.

    • Problems

  • there are 3 major concerns for this problem
    • how to store metadata for 64-bits-id ledgers in zookeeper?
    • how to generate 64-bits ledger id using zookeeper?
    • how to achieve backward compatibility?
  • how to store?
    • Salesforce built a Long Hierarchical Ledger Manager
    • Ledger ID is split in 3-4-4-4-4 components.
    • No backward compatible on initial built
    • backward compatibility?
      • for ledger id whose higher 32 bits are zero, use split 2-4-4
      • for ledger id whose higher 32 bits are higher than zero, use split 3-4-4-4-4
      • since the first component has different characters (2 vs 3), it is easy to achieve backward compatibility
    • negative ledger id discussion

  • how to generate?

    • use zookeeper sequence znode to create lower 32 bits

    • use another znode to create sequential buckets, the buckets are used for higher 32 bits.
    • when to create bucket?
      • when lower 32-bits generation is approaching 31 bits, create a new bucket.
      • creating a new bucket and the id generation will happen under one zookeeper 'multi' transaction call to guarantee there is only one bucket is created.

  • 128 bits

  • we also discussed 128 bits support. but it will touch a lot - interface, wire protocol and storage

    • we decided to move to a separated discussion