Attendees:

NOTES

  • Discussed BP-2: Resource Placement

    • Disk Capacity as weight. Can be extended to other metrics.

    • This feature can be enabled/disabled by a configuration flag

    • Client will query all the bookies to build the weight information

      • New protocol message

      • Watch for bookies removal and additional

    • Most of the logic is on client side, not on the bookies

    • Hotspot: (pick the solution 2)

      • Solution 1: limit on N * min_weight

      • Solution 2: limit on N * median_weight

      • Solution 3: another dimension of weight (not going because of the get bookie info storm)

    • Protocol : Get bookie info request

      • There will be a connection concern when the number of clients and number of bookies grows.

      • Gossip Protocol for collecting information

    • AutoRecovery

      • AutoRecovery is using client which it would automatically leverage the change.
    • Implementation

      • UpdateBookieInfo - recalculating the weights

      • Rackware - no change to select rack logic

      • BookieInfoReader - Singleton per bookkeeper object

    • Stats and Monitoring

    • Rollout Plan

  • Questions

    • BookieInfo (Enrico Olivelli)

      • Add label into the bookie info for multi-tenant placement

      • A generic properties

    • AutoRecovery (Yiming Zang)

      • Two different ways to auto recovery

      • Running autorecovery with bookies
      • Running autorecovery separatedly

    • Rack (Yiming Zang)

      • How to handle weighted rack placement? 
      • Propagated the weight information through the whole NetworkTopology

    • Connections

      • Gossip

      • Random backoff to avoid request storm

    • Alternatives

      • Zookeeper based

      • Let's add tickets to track enhancements

    • BookieInfoReader

      • Lazy initialization for write ledger handle

  • No labels