A recording of the webinar about Apache Software Process:
IMPORTANT
The JIRA handling process outlined below should be followed in absolutely all cases, without exceptions, regardless of the ticket complexity.
JIRA Accounts
New contributors should register account at https://issues.apache.org/jira/secure/Dashboard.jspa and send out email to project's dev list with request for contributors permissions. Committers should handle this request and grant contributor access to new community member.
Tickets are picked up by community members from a pool of unassigned and unscheduled tickets after discussion on project's dev list. Assigning tickets to a version contributor helps others to understand what will be included in next release.
JIRA issues are grouped by fixVersion
field, which is an intended version of the product feature gets merged to. Everyone should make sure that after closing fixVersion
is set to a proper version that actually releases the changes made in scope of the ticket.
The following page contains information on upcoming releases (draft) Release Plan.
IN PROGRESS
state.If the changes implemented under the ticket require changes in user documentation, create a related documentation ticket (add "Documentation" to the Component field) and provide a reasonable amount of details in the ticket's description. The information provided in the ticket should be sufficient for any contributor to start working on it. If there is no need to change user documentation, uncheck the Docs Required flag. The Docs Required flag is used to filter out the tickets that require documentation so that our documentation is always up to date.
Master Branch
Changes to a RTC module should be reviewed at least by one maintainer for that module (which may or may not be the same as the main reviewer) before being merged.
Changes to a CTR module (CTR*) may be merged without waiting for review by a maintainer. However it's advisable to ask a maintainer of a CTR module for review and to wait for some time for his/her feedback.
Component | Maintainers |
---|---|
Ignite Core (data grid, rebalancing, affinity, the rest of internals not covered below) | Semen Boikov, Alexey Goncharuk, Yakov Zhdanov, Anton Vinogradov |
Marshalling (Binary, Optimized, JDK) | Vladimir Ozerov, Denis Magda, Alexey Goncharuk |
Discovery & Communication SPIs | Yakov Zhdanov, Semen Boikov, Denis Magda |
Ignite Compute API | Yakov Zhdanov, Valentin Kulichenko |
Ignite Services API | Valentin Kulichenko |
Ignite SQL & Text Queries | Sergey Vladykin |
Ignite Continuous Queries | Nikolai Tikhonov, Semen Boikov |
Build System | Anton Vinogradov |
Hadoop Accelerator | Vladimir Ozerov, Ivan Veselovsky |
Spark Shared RDDs | Alexey Goncharuk, Andrey Gura |
IGFS | Vladimir Ozerov, Ivan Veselovsky |
.Net API | Pavel Tupitsyn, Vladimir Ozerov |
C++ API | Igor Sapego, Vladimir Ozerov |
Streamers (JMS, Flume, Kafka, etc.) CTR* | Raúl Kripalani, Roman Shtykh |
Docker, Mesos, YARN integration | Nikolai Tikhonov |
AWS, Google Compute Engine, JClouds integration | Denis Magda, Nikolai Tikhonov |
OSGi integration | Raúl Kripalani, Denis Magda |
Visor | Alexey Kuznetsov |
WebSession & WebSession Filter | Valentin Kulichenko |
PATCH AVAILABLE
state.Optional: Tips to pass review quickly
Ask commiter to review changes directly.
Check affected files git history to find person most likely able to review changes.
In case it's hard to determine who's able to review by git history use maintainers list presented above.
Add "review request" comment to the Jira Issue starting with a commiter username.
for example: "[~avinogradov], Please review my changes."
Commiter will gain notification and review your changes and/or help to find another commiter to do this.
Upsource is an online code review tool. It provides a convenient way to view and discuss changes.
Upsource is optional: it is up to author and reviewer whether to use it or not. Big changes with long discussions are easier to manage in Upsource.
Code author workflow:
Reviewer workflow:
Apache Ignite community agreed to release new version at least once per quarter. However, duration may be longer or shorter. After development of new functionality is finished, QA cycle starts, then release procedure follows.
Master
should become the development branch for the next release.master
. This way master
can be used to develop functionality of the next release. All release fixes get merged to release branch and then to master
.master
branch (or release branch if one exists - in this case changes get merged to release branch and then to master
branch). Changes get merged to master
branch of the project Git via process described at Workflow.master
(or release) branch.Normally, project repo should contain only master
branch, very few branches for ongoing releases and commiter's branches ready to be reviewed. Committers and PMC members are in charge to make everyone follow this rule.
Instructions on how to build source and binary releases can be found at DEVNOTES.txt
. Please see "Ignite Release Instructions"
section. On how to make official release please refer to Release Process.
There are 3 way how you can make contribution
+------------+ +---------------+ +-----------------+
| | replica | | fork | |
| Apache Git | ==========> | GitHub Mirror | ---------> | John Doe's Fork |
| | | | | |
+------------+ +---------------+ +-----------------+
^ ^ ^
| | |
| +------------------------+ | origin
| upstream | |
| | |
| +-----------------+
| *Apache Git remote handle for committers* | |
+------------------------------------------------| Local clone |
| |
+-----------------+
To start:
You will need to update a local master sometimes (to merge to your development branches sometimes). How to do it:
Add remote for Apache Ignite mirror (you need to do it once)
git remote add upstream https://github.com/apache/ignite.git
Each time when you want to update your local master do the following:
git pull upstream git checkout master
To make contribution:
In additional to contributors configuration, committers need to have one more remote reoi - for working with Apache Git repo. It can be added like this:
git remote add apache https://git-wip-us.apache.org/repos/asf/ignite
To push any branch at Apache repo use
git push apache <branch_name>
To apply a pull-request it's strongly recommended using ./scripts/apply-pull-request.sh script. Script takes 'pull-request-id' as a parameter and do next:
git fetch upstream pull/<id>/head:pull-<id>-head
git merge --squash pull-<id>-head
git commit --author=“<saved_author>" -s -m “<comment> - Fixes #<id>.”
Now, you will have one commit at master with all changes from pull-request. Changes can be reviewed again. If you accept all changes and want to push it, do next:
git push apache master
You can start by cloning the Ignite GIT repo.
Clone the repo
git clone https:
//git-wip-us
.apache.org
/repos/asf/ignite
We use git as our version control system. To streamline the process of giving proper credit to the contributors when committing patches, we encourage contributors to submit patches generated using "git format-patch"
command. This has many benefits:
Long story short, it makes both the contributors' and committers' lives easier, so please generate your patches using git format-patch
.
We have the following requirements for patches:
We prefer that you use the following step-by-step instructions when developing with Ignite.
For example, if you are starting working on the feature IGNITE-9999.
## Get the repo git clone https://github.com/apache/ignite.git ## Some development here with many commits at ignite-9999. git commit -a -m 'ignite-9999: Intermediate commit 1'; ... git commit -a -m 'ignite-9999: Intermediate commit 100'; ## Commit the last changes. git commit -a -m 'ignite-9999: Implemented.'; ## Making patch. ## There are a lot of changes at ignite-sprint-999 and we need to get it, resolve conflicts (if exists), rerun tests for ignite-9999. git checkout master git pull git checkout ignite-9999 git merge master ## Run script to make patch. Patch will have all changes as one commit. <ignite_home>/scripts/git-format-patch.sh
Note: it is strongly recommended to merge 'master' branch to your development branch, for example, every day (or after each commit).
Created patch-file should be attached to a JIRA ticket and the ticket status should be changed on Patch Available.
If you do everything correctly, then all necessary TeamCity test builds will be triggered automatically in 3 minutes period, and a comment with triggered build information (test package names, TeamCity build links) will be added to the JIRA.
Once tests are passed, the patch can be reviewed and merged by a committer.
Requirements (to auto triggering the test builds):
Contributor patches have to be applied by next command.
git am -s <patch-file>
This command apply a patch file generated by 'git format-patch', stores information who created a patch and who applied a patch. So, it gives proper credit to the contributor and store an information who decide to take these changes.
If patch file has been created by scripts/git-format-patch.sh then a name of a patch-file contains a short hash revision of master branch revision against which patch has been created. Patch file template is
master_<hash>_ignite-<ticket-number>.patch
For example
master_b001525_ignite-9999.patch
So, in case, if a patch can't be applied without conflicts on the HEAD of master and the patch has been created by scripts/git-format-patch.sh then a commiter can apply the patch to master by a revision hash, review changes and resolve the conflicts by yourself.
Whenever working on bigger features, committers can also create 'ready to be reviewed' branch ignite-XXXX, where XXXX is the number of the JIRA ticket.
TeamCity should be forced to run all tests on created branch before review. Once tests are passed, the branch has be reviewed by module's maintainer.
Created branch name should be attached to a JIRA ticket and the ticket status should be changed on Patch Available.
Branch can be merged to master on sucessful review by at least one another committer.
Branch should be deleted on branch merged to master or issue cancelled. Committers are in charge of deleting their branches.
List of points should be checked before push:
Make sure project build log contains no javadoc warnings. Grep build output for "Javadoc Warnings". Covered by Licenses & Javadoc TeamCity task.
In case a new module is added, make sure it contains README.txt at the module's root.
If the contribution is significant (new feature or significant rework of an existed functionality or API) make sure that an example is added to 'ignite-example' and/or an article is written for Apache Ignite Documentation.
Since readme.io does not automatically copy the changes from the current version to the subsequent version, documentation for any new feature that will be released in the next version should be created within the document for the current version. These new pages should be kept hidden until the next version released.
In case new module added, make sure source and binary distributions contains correct license files at modules folders. Covered by Licenses & Javadoc TeamCity task.
Can be checked manually by running command
mvn clean validate -DskipTests=true -P check-licenses
Make sure each package contains package-info.java with proper description.
Make sure that command
mvn clean package -DskipTests
finishes without errors.
Apache Way
A recording of the webinar about Apache Software Process:
JIRA Process
The JIRA handling process outlined below should be followed in absolutely all cases, without exceptions, regardless of the ticket complexity.
New contributors should register account at https://issues.apache.org/jira/secure/Dashboard.jspa and send out email to project's dev list with request for contributors permissions. Committers should handle this request and grant contributor access to new community member.
Getting Started
Easy tickets to get started with Apache Ignite: easy tickets
All other issues: Ignite Jira
Tickets and Versions
Tickets are picked up by community members from a pool of unassigned and unscheduled tickets after discussion on project's dev list. Assigning tickets to a version contributor helps others to understand what will be included in next release.
JIRA issues are grouped by fixVersion field, which is an intended version of the product feature gets merged to. Everyone should make sure that after closing fixVersion is set to a proper version that actually releases the changes made in scope of the ticket.
The following page contains information on upcoming releases (draft) Release Plan.
Ticket Creation
Every JIRA ticket should be sufficiently described.
There should not be any tickets with title only and without description.
If there is a discussion pertaining to the ticket, the ticket should have a link to the dev list.
Working on a ticket
Anyone in the community can start working on any unassigned ticket.
(OPTIONAL) It is recommended to clarify the relevance and correctness of the ticket in the community before doing actual implementation. To do this, you can write an email to the dev.list and wait for answer.
Before beginning working on a ticket, you should assign the ticket to yourself.
Move the ticket to IN PROGRESS state.
If necessary, add comments describing the design decisions or approach you plan to take.
Review Process and Maintainers
Ignite employs both Review-Then-Commit (RTC) process for most of the components and Commit-Then-Review (CTR) for few ones.
Please consult to Review Checklist to understand how tickets are reviewed and what rules to follow.
Ignite "master" branch should always be release-ready. Please avoid any commits or merges to the "master" branch unless the whole TeamCity CI suite has passed.
Changes to a RTC module should be reviewed at least by one maintainer for that module (which may or may not be the same as the main reviewer) before being merged.
Changes to a CTR module (CTR*) may be merged without waiting for review by a maintainer. However it's advisable to ask a maintainer of a CTR module for review and to wait for some time for his/her feedback.
Component Maintainers
Ignite Core (data grid, rebalancing, affinity, the rest of internals not covered below) Semen Boikov, Alexey Goncharuk, Yakov Zhdanov, Anton Vinogradov
Marshalling (Binary, Optimized, JDK) Vladimir Ozerov, Denis Magda, Alexey Goncharuk
Discovery & Communication SPIs Yakov Zhdanov, Semen Boikov, Denis Magda
Ignite Compute API Yakov Zhdanov, Valentin Kulichenko
Ignite Services API Valentin Kulichenko
Ignite SQL & Text Queries Sergey Vladykin
Ignite Continuous Queries Nikolai Tikhonov, Semen Boikov
Build System Anton Vinogradov
Hadoop Accelerator Vladimir Ozerov, Ivan Veselovsky
Spark Shared RDDs Alexey Goncharuk, Andrey Gura
IGFS Vladimir Ozerov, Ivan Veselovsky
.Net API Pavel Tupitsyn, Vladimir Ozerov
C++ API Igor Sapego, Vladimir Ozerov
Streamers (JMS, Flume, Kafka, etc.) CTR* Raúl Kripalani, Roman Shtykh
Docker, Mesos, YARN integration Nikolai Tikhonov
AWS, Google Compute Engine, JClouds integration Denis Magda, Nikolai Tikhonov
OSGi integration Raúl Kripalani, Denis Magda
Visor Alexey Kuznetsov
WebSession & WebSession Filter Valentin Kulichenko
Submitting for Review
Attach a patch, pull request url or branch name (see instructions at Workflow)
(OPTIONAL) create an Upsource review: Review With Upsource
Add comment describing what has been implemented.
Move ticket to PATCH AVAILABLE state.
Ask commiter to review changes directly.
Check affected files git history to find person most likely able to review changes.
In case it's hard to determine who's able to review by git history use maintainers list presented above.
Add "review request" comment to the Jira Issue starting with a commiter username.
for example: "[~avinogradov], Please review my changes."
Commiter will gain notification and review your changes and/or help to find another commiter to do this.
Reviewing a Ticket
Make sure that your patch satisfies Review Checklist rules.
Each comment should be started with [~username] to guarantee proper notification.
Commiter should add comment like "Changes accepted and ready to be merged." once review successfully finished.
If the commiter/contributor has proposals, it is recommended to add a comment in the ticket in addition to PR / CR comments referring proposals added. This helps other committers to identify that fix has something to improve.
Closing a Ticket
Once ticket has passed all the reviews and has no additional comments, a committer should apply the latest patch to the master branch.
A comment should be added to the ticket stating that the patch has been applied to master.
Move ticket to RESOLVED state.
Review With Upsource
Upsource is an online code review tool. It provides a convenient way to view and discuss changes.
Upsource is optional: it is up to author and reviewer whether to use it or not. Big changes with long discussions are easier to manage in Upsource.
Code author workflow:
Register on http://reviews.ignite.apache.org/
Go to Branches view: http://reviews.ignite.apache.org/ignite/branches
Click on your pull request ("PR XXXX"). Upsource can take up to 1 minute to discover new pull request.
Click "Create branch review"
Rename review to include JIRA key and description (example: "IGNITE-42 Support CacheLoader and CacheWriter")
In JIRA, use More -> Link -> Web Link to link the review page
New changes in the pull request are picked up automatically (may take a minute)
Reviewer workflow:
Follow the Upsource link in JIRA
Select files on the left to view changes
Click on line numbers to leave comments
Leave general comments on the timeline screen
Addressed comments may be marked as "Resolved"
When all issues are resolved (if any), click "Accept" on the top left, and leave a comment in JIRA
Git Process
Apache Ignite community agreed to release new version at least once per quarter. However, duration may be longer or shorter. After development of new functionality is finished, QA cycle starts, then release procedure follows.
Overview
Master should become the development branch for the next release.
Whenever current release goes to QA, new branch should be created from master. This way master can be used to develop functionality of the next release. All release fixes get merged to release branch and then to master.
All individual ticket/features development happen in separate repositories (either local or forks of GitHub mirror - https://github.com/apache/ignite) . Firm rule: ticket/feature branches should never be pushed to repo.
Development branches should be created off of the master branch (or release branch if one exists - in this case changes get merged to release branch and then to master branch). Changes get merged to master branch of the project Git via process described at Workflow.
Git tag should be created for every release.
All CI tests must pass before the merge of ticket branches to the master (or release) branch.
Normally, project repo should contain only master branch, very few branches for ongoing releases and commiter's branches ready to be reviewed. Committers and PMC members are in charge to make everyone follow this rule.
Release Process
Instructions on how to build source and binary releases can be found at DEVNOTES.txt. Please see "Ignite Release Instructions" section. On how to make official release please refer to Release Process.
Workflow
There are 3 way how you can make contribution
GitHub pull-request
Patch-file
Ticket Branch (only committers)
1. Create GitHub pull-request
+------------+ +---------------+ +-----------------+
| | replica | | fork | |
| Apache Git | ==========> | GitHub Mirror | ---------> | John Doe's Fork |
| | | | | |
+------------+ +---------------+ +-----------------+
^ ^ ^
| | |
| +------------------------+ | origin
| upstream | |
| | |
| +-----------------+
| *Apache Git remote handle for committers* | |
+------------------------------------------------| Local clone |
| |
+-----------------+
Creation
To start:
You need to fork an Apache Ignite mirror on GitHub.
Make a local copy of your Apache Ignite mirror fork. Your remote origin will refer to 'https://github.com/<your_github_uname>/ignite.git'.
You will need to update a local master sometimes (to merge to your development branches sometimes). How to do it:
Add remote for Apache Ignite mirror (you need to do it once)
git remote add upstream https://github.com/apache/ignite.git
Each time when you want to update your local master do the following:
git pull upstream
git checkout master
To make contribution:
Fix / implement JIRA ticket in your fork. Provide Java docs whenever required. If you add a new package make sure that package-info.java file in it is in place with a description. Commit branch to origin (origin = your fork). It's recommended to develop IGNITE-xxx ticket at ignite-xxx branch.
If your contribution is significant (new functionality, deeply reworked existed functionality API) then add an example of usage to 'ignite-example' and add an article to Apache Ignite Documentation.
Create pull request from the new remote branch in the fork to master of Apache Ignite mirror. Please, start a title of the pull request from 'IGNITE-xxx'. An email about the pull request will be send to dev-list and the same JIRA comment will be added to the IGNITE-xxx ticket.
Trigger validation of those test suites that have been affected by your changes on TeamCity:
Locate a test suite you have to check, press button named "..." that is located on the left of "Run" button. "Run custom build" dialog will appear;
Go to "Changes" tab and choose "pull/<pull-request-number>/head" in "Build branch" dropdown list;
Press "Run build" button and monitor tests results.
Once tests are passed, the pull request can be reviewed and merged by a committer. Move a corresponding JIRA ticket to "Patch Available" state by clicking on "Submit Patch" button and let the community know that you're ready for review.
Note: Existing pull request should be updated instead of creation of new one. Creation of more than one pull request for one issue forbidden.
Applying
In additional to contributors configuration, committers need to have one more remote reoi - for working with Apache Git repo. It can be added like this:
git remote add apache https://git-wip-us.apache.org/repos/asf/ignite
To push any branch at Apache repo use
git push apache <branch_name>
To apply a pull-request it's strongly recommended using ./scripts/apply-pull-request.sh script. Script takes 'pull-request-id' as a parameter and do next:
Checks that you don't have any uncommitted changes.
Checks you are one master branch and master branch is up-to-date.
Updates local master from Apache git repo.
Fetches pull request to a local branch:
git fetch upstream pull/<id>/head:pull-<id>-head
Saves an author and a comment of the last commit at pull-<id>-head.
Merges from the new branch to master:
git merge --squash pull-<id>-head
Ask you about custom comment or using the saved comment.
Commit to local master. The script automatically sign-off a commit and add "Fixes #<id>." suffix to comment (It will close the pull request, see https://help.github.com/articles/closing-issues-via-commit-messages/):
git commit --author=“<saved_author>" -s -m “<comment> - Fixes #<id>.”
Now, you will have one commit at master with all changes from pull-request. Changes can be reviewed again. If you accept all changes and want to push it, do next:
git push apache master
2. Create a Patch-file
Creation
You can start by cloning the Ignite GIT repo.
git clone https://git-wip-us.apache.org/repos/asf/ignite
We use git as our version control system. To streamline the process of giving proper credit to the contributors when committing patches, we encourage contributors to submit patches generated using "git format-patch" command. This has many benefits:
Committers will not forget to give proper credit to a contributor
Contributor's name and email address show up in the git log
Long story short, it makes both the contributors' and committers' lives easier, so please generate your patches using git format-patch.
We have the following requirements for patches:
patch can be applied to the HEAD of 'master' branch by "git am <patch-file>" without conflicts.
patch has to have all changes in one commit with comment like "ignite-xxx: Implemented."
patch-file has to have 'patch', 'txt' or 'diff' extension
Patch Creation Process
We prefer that you use the following step-by-step instructions when developing with Ignite.
For example, if you are starting working on the feature IGNITE-9999.
## Get the repo
git clone https://github.com/apache/ignite.git
## Some development here with many commits at ignite-9999.
git commit -a -m 'ignite-9999: Intermediate commit 1';
...
git commit -a -m 'ignite-9999: Intermediate commit 100';
## Commit the last changes.
git commit -a -m 'ignite-9999: Implemented.';
## Making patch.
## There are a lot of changes at ignite-sprint-999 and we need to get it, resolve conflicts (if exists), rerun tests for ignite-9999.
git checkout master
git pull
git checkout ignite-9999
git merge master
## Run script to make patch. Patch will have all changes as one commit.
<ignite_home>/scripts/git-format-patch.sh
Note: it is strongly recommended to merge 'master' branch to your development branch, for example, every day (or after each commit).
Patch Validation
Created patch-file should be attached to a JIRA ticket and the ticket status should be changed on Patch Available.
If you do everything correctly, then all necessary TeamCity test builds will be triggered automatically in 3 minutes period, and a comment with triggered build information (test package names, TeamCity build links) will be added to the JIRA.
Once tests are passed, the patch can be reviewed and merged by a committer.
Requirements (to auto triggering the test builds):
patch-file has to have 'patch', 'txt' or 'diff' extension
patch-file has to be attached by JIRA user with contributor role.
Note: All TeamCity test builds can be triggered manually via "Ignite/ -> Run All for patch" (by 'Jira number'). A comment will be added to the jira with all new triggered builds.
Applying
Contributor patches have to be applied by next command.
git am -s <patch-file>
This command apply a patch file generated by 'git format-patch', stores information who created a patch and who applied a patch. So, it gives proper credit to the contributor and store an information who decide to take these changes.
If patch file has been created by scripts/git-format-patch.sh then a name of a patch-file contains a short hash revision of master branch revision against which patch has been created. Patch file template is
master_<hash>_ignite-<ticket-number>.patch
For example
master_b001525_ignite-9999.patch
So, in case, if a patch can't be applied without conflicts on the HEAD of master and the patch has been created by scripts/git-format-patch.sh then a commiter can apply the patch to master by a revision hash, review changes and resolve the conflicts by yourself.
3. Create a Ticket Branch (only committers)
Creation & Applying
Whenever working on bigger features, committers can also create 'ready to be reviewed' branch ignite-XXXX, where XXXX is the number of the JIRA ticket.
TeamCity should be forced to run all tests on created branch before review. Once tests are passed, the branch has be reviewed by module's maintainer.
Created branch name should be attached to a JIRA ticket and the ticket status should be changed on Patch Available.
Branch can be merged to master on sucessful review by at least one another committer.
Branch should be deleted on branch merged to master or issue cancelled. Committers are in charge of deleting their branches.
Checklist before push
List of points should be checked before push:
Javadoc
Make sure project build log contains no javadoc warnings. Grep build output for "Javadoc Warnings". Covered by Licenses & Javadoc TeamCity task.
Readme
In case a new module is added, make sure it contains README.txt at the module's root.
Documentation & Examples
If the contribution is significant (new feature or significant rework of an existed functionality or API) make sure that an example is added to 'ignite-example' and/or an article is written for Apache Ignite Documentation.
Since readme.io does not automatically copy the changes from the current version to the subsequent version, documentation for any new feature that will be released in the next version should be created within the document for the current version. These new pages should be kept hidden until the next version released.
Licenses
In case new module added, make sure source and binary distributions contains correct license files at modules folders. Covered by Licenses & Javadoc TeamCity task.
Can be checked manually by running command
mvn clean validate -DskipTests=true -P check-licenses
package-info.java
Make sure each package contains package-info.java with proper description.
Project Build
Make sure that command
mvn clean package -DskipTests
finishes without errors.