• Created by Unknown User (jujjuri), last updated by Unknown User (eolivelli) on Apr 20, 2017 2 minute read

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 11 Next »

Attendees:

  • JV, Bobby, Kishore, Matteo, Francesco, Zhimerg, Sam, Enrico, Charan, Yiming

 Agenda:

  • Discuss about Netty 4
  • Discuss: Write/AddEntry failures beyond Ack Quorum responses are ignored by the client. Replication scans to detect these are expensive and NOT bullet proof. It just scans once a week, and just checks first and last entry in the segment, which may not be enough. Especially the case if bookie is up but certain ledger dir on the bookie got deleted or some other scenario where a under replicated hole can go undetected.
  • Testing: Existing Junit tests doesn't come close to give confidence for any distributed storage systems. What kind of destructive, consistency, and constraint guarantee tests are done at Twitter/Yahoo? As a community what else we can do? Jepsen? Runway? What does community think? how can we improve this area?
  • Discuss next release plan.

  • BOOKKEEPER-1040) Use separate log for compaction - Salesforce used different techniques to address this problem, we can discuss.

  • Salesforce started looking into having multiple entrylogs.

 

Open JIRA issues for 4.5.0:

Key Summary T Created Updated Due Assignee Reporter P Status Resolution
Refresh

 

Open Pull Requests:e

https://github.com/apache/bookkeeper/pulls

 

Discussion:

 

  • Netty 4: problems with pause/restore handlers. Kishore will look into Yahoo/Matteo changes in order to get tests working. There is some flaky test which is failing due to lost entries/acks (expectially the SlowBookieTest)
  • Replicator: problem with long lived ledgers. We are now aware of lost entries due to corrupted disks. No way to detect failures.
    • (Yiming) Twitter: lost-disk -> drop the bookie -> the replicator will substitute the bookie
  • JV: Tests frameworks for corruption/failures:
    • Twitter: dedicated clusters for tests, inject failures at pub/sub level and at SO/TCP level
    • Yahoo: inject failures at pub/sub level
    • MagNews:  inject failures at application level
  • Sam at Salesforce: fault injection at appication level. Create dedicated clusters of machine to run system/integrations tests. Sam started working on 
  • JV: I wonder if anyone could volounter to study Jepsen and Runway to use it for BookKeeper
  • Bobby: it just a matter of resources. In the short term we have to finish the 'pushback'
  • Enrico will look at it, let's start a mailing list thread

 

  • No labels