THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
The Build Lead role is a volunteer role with weekly rotations.
Date Range (reversed) | Week | Name | #cassandra-dev slack | |
---|---|---|---|---|
- | 21 | |||
- | 20 | |||
- | 19 | |||
- | 18 | |||
- | 17 | Mick Semb Wever | mck@apache.org | mck |
- | 16 | Dan Jatnieks | djatnieks@gmail.com | Dan Jatnieks |
- | 15 | |||
- | 14 | Josh McKenzie | jmckenzie@apache.org | jmckenzie |
- | 13 | Josh McKenzie | jmckenzie@apache.org | jmckenzie |
- | 12 | Mick Semb Wever | mck@apache.org | mck |
- | 11 | maxwellguo | cclive1601@gmail.com | maxwellguo |
- | 10 | Derek Chen-Becker | apache@chen-becker.org | dchenbecker |
- | 9 | Derek Chen-Becker | apache@chen-becker.org | dchenbecker |
- | 8 | Mick Semb Wever | mck@apache.org | mck |
- | 7 | maxwellguo | cclive1601@gmail.com | maxwellguo |
- | 6 | German Eichberger | geeichbe@microsoft.com | xgerman |
- | 5 | Dan Jatnieks | djatnieks@gmail.com | Dan Jatnieks |
- | 4 | Claude Warren | claude.warren@aiven.io | Claude Warren |
- | 3 | Caleb Rackliffe | calebrackliffe@gmail.com | Caleb Rackliffe |
- | 2 | Mick Semb Wever | mck@apache.org | mck |
See child pages for past years.
...
- Check if there are new test failures in Butler that don't yet exist in JIRA (i.e. butler test failures w/out a JIRA link)
- Create JIRA tickets for new failures and link them to the failure entries in Butler
- Keep an eye out for major build infra issues, if some show up hit #cassandra-dev about it
- Optional:
- Assign test failure JIRA to whomever introduced a new failing test or, if clear, broke an existing stable
- test
- Keep an eye on any build infra issues, make sure there are JIRAs created, send email to dev@ notifying about status and progress.
- (Optional): run a hires config against trunk / other desired branches on circleci, confirm tickets created for failures, create tickets if none
Details
Creating JIRAs
- Create a JIRA ticket with summary: "Test Failures: <suite> <class_name>"
- Set component to the matching "Test/<suite>" component
Fill out description w/mention of class name and number of failures at time of ticket creation
In comments, add details of failure w/link to failing run + formatted \{code\} (without \) blocked JIRA capturing output of the test as CI results aren't preserved forever
- After creation, update the ticket to Bug Category "Correctness", "Test Failure"
...
- Check for new failures on the details page for each branch in the bottom right where it says detailed history:
- Look for failing tests without a JIRA link; in the following example see the top test "TestCQLNodes2RF1_Upgrade_current_4_0_x_To_indev_trunk:
- For failing tests without a linked item we have a couple workflows depending on where the commit occurred as well as what type of failure it is:
- Single commit on trunk:
- If intermittent, create a new JIRA ticket w/"intermittent failure" in the summary for the failure and link it in Butler
- If consistent, git revert the SHA that introduced the failure, re-open the original JIRA ticket, and leave a note for the original assignee about the breakage they introduced.
- Commit on older LTS branch w/merge commits:
- If intermittent, create a new JIRA ticket w/"intermittent failure" in the summary for the failure and link it in Butler
- If consistent, create a new JIRA ticket for the failure, link it in Butler, and set assignee to the individual that introduced the failure and notify them in the comments in the JIRA ticket
- Single commit on trunk:
Build infra:
...
- If there are any build failures due to infra issues (say running out of disk space on Jenkins) either from the weekly cci run or when checking Butler problems, file or find existing JIRA
Notes:
- Link failures to JIRA via the "Link selected failures" button:
- Create new failure tickets in the ASF C* JIRA.
- CI on Jenkins is run on every commit so for consistently failing tests (> 1 run failed on butler) it should be immediately clear which commit introduced the failure.
- For failures with "Timeout occurred. Please note the time in the report does not reflect the time until the timeout", we can ignore them , as it's considered test-infrastructure failures. And CASSANDRA-18137 is working on this kind of failures already.
- [Optional] Loop failing tests locally using tools/dev/ci-test-loop (PENDING CONTRIBUTION), which relies on tools/dev/ci-test (PENDING CONTRIBUTION) for a number of iterations to determine if it's consistent or intermittent. If intermittent, reflect in subject of the created JIRA ticket for the failure.CI on Jenkins is run on every commit so for consistently failing tests (> 1 run failed on butler) it should be immediately clear which commit introduced the failure.