THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
Retrospective
What did we do well?
- Ewen Cheslack-Postava
- Mostly respected the deadlines
- Didn't have any major complaints about features slipping to next release – with the time based release folks seem to be ok with missing the deadline if they really have missed it.
- Shipped a ton of great features and bug fixes, and the RC process worked well by catching a number of important bugs.
What should we have done better?
- Ewen Cheslack-Postava
- Deluge of KIPs and PRs for KIPs at the last minute. I think we should consider adding another cutoff deadline for these. If a KIP is being voted in very last minute, then there's not enough time to properly review the code.
- Vast majority of my time as release manager was spent just tracking JIRAs/clearing them out of the release. I'd like to see release management become a lot more mechanical. I have scripted some of the process of running a release, but that won't help with this stage. I think the thing that will help the most with this is if people don't tag JIRAs as targeted for a specific release until they a) have the PR and just need review or b) we know it's a blocker for that release.
- Organizing the summary of changes was a bit tough – I had to base it mostly on KIPs because there are too many JIRAs to sift through. Perhaps including a section for this in the release plan template and asking committers in each area to contribute notes would help with this.
- I have scripted generating RCs, would be nice to script the final release steps as well. However, some of these can only be done by a PMC member. Perhaps scripting them would make it easier to just pass the final release steps to any PMC member instead of having the RM do half of the steps and the PMC member does the rest.
- I think at least one of the blocker bugs probably could have been caught by stress testing earlier. Could we either a) encourage those doing testing to do it more regularly on nightlies or b) maybe get some feedback/contributions to the system tests so that the AK project itself will catch these? If b) the most useful feedback from the community is what tests they find have been most valuable in catching issues since AK has limited bandwidth for adding new tests.