Excerpt | ||
---|---|---|
| ||
IMPLEMENTED |
Jira | ||||||||
---|---|---|---|---|---|---|---|---|
|
Overview
We will split current SVN repository located at https://svn.apache.org/repos/asf/sling/trunk/ into multiple git repositories hosted on the ASF git servers. We will opt in to the Git "Dual Master" system which will allow us to push directly to Github. Direct access to Github and the possibility of directly merging pull requests can be a booster for community participation.
We will not enable other Github features, such as wikis, features, and projects.
The rule of the thumb for selecting what ends up in a git repository is a releasable Maven unit. The naming of git repositories will be sling-${sanitizedArtifactId
} . The santizied artifactId is composed from the Maven artifact id where all dots are replaced by slashed. See
Jira | ||||||||
---|---|---|---|---|---|---|---|---|
|
Commit history
Commit history will be preserved as we will convert the projects using git-svn, followed by git filter-branch. However, this has limitations when the project has been moved across multiple directories. We have detected it so far for the repoinit modules. We consider this acceptable, as the full history also resides in SVN. There are no confirmed alternatives that can produce better results. Trying to import the whole ASF SVN repository to experiment with the reposurgeon tool is taking too much time - still not done after five days.
Special situations
The following directories will not be split to individual modules:
- tooling/ide - the modules are not individually buildable and releasable
- performance - the modules are tied together in a performance testing harness
- samples - having samples spread over multiple repositories will make it harder for users to discover all of them
- whiteboard - since we intend to perform no releases from the whiteboard, it does not make sense to create individual repositories. Also, personal Github repositories can be used as well, given that the owner is an Apache committer and only they contribute to that repository.
We will not migrate the attic as there will be no further work done on the modules in there.
The following non-Maven directories will be migrated as separate repositories
- tooling/scm
- tooling/jenkins
Unified view of the repository
We will create the special 'repo' sling which will contain one or more repo
manifests which will allow viewing and editing multiple repositories at once.
Canonical repository URLs
As part of the gitbox setup, the repositories can be accessed both via GitHub and gitbox.apache.org . Since there is a strong requirement to cut releases from ASF servers, all pom references will point to gitbox instead of GitHub. Of course, committers are free to use GitHub as a remote for pushing, just releases are affected.
Migrating a module
If an individual repository needs to be migrated, usually because it was missed from the initial migration, the following steps are needed (assuming we need to migrate contrib/scripting/esx
:
Code Block |
---|
$ git clone https://github.com/apache/sling-old-svn-mirror # clone the old svn which will be used as a source for the migration
$ cd sling-old-svn-mirror
$ echo contrib/scripting/esx > repo-candidates.txt # file name is not important, but it's helpful to have the name(s) stores in a central place
$ ./tooling/scm/scripts/migrate-to-git.sh -c < repo-candidates.txt # converts the repositories locally in ../sling-modules/
$ ./tooling/scm/scripts/migrate-to-git.sh -r < repo-candidates.txt # provisions the git repository on github/gitbox . Github permissions take 30-60 minutes to sync, you will not be able to push to this repo immediately
$ cp LICENSE ../sling-modules/sling-org-apache-sling-scripting-esx/ && cd ../sling-modules/sling-org-apache-sling-scripting-esx/ && git add LICENSE && git commit -m 'Added LICENSE file' && cd -
$ ./tooling/scm/scripts/migrate-to-git.sh -p < repo-candidates.txt # after verifying that the repository is writable on github
|
Once the repository is provisioned add it to the sling-aggregator repo by following the instructions from the aggregator README.
Older information
Tools and scripts
Some WIP tools can be found at https://svn.apache.org/repos/asf/sling/trunk/tooling/scm/scripts/ .
Can Sling (committers and contributors) benefit from moving to Git?
Pros
- offline support
- cherry picking
- easy to move projects (e.g. between bundles, contrib, whiteboard...)
- adpoting the Github workflow using the Git Dual Master system
- ...
Cons
- not eating our own dog food
- slow for big repositories (many modules)
- tagging is global
- ...
What needs to be prepared for moving to Git?
- repository layout
- release process
- documentation
- CI
- ...
Git at Apache
http://wiki.apache.org/general/GitAtApache
http://www.apache.org/dev/git.html
Repository Layout and Projects (Modules)
There are currently 250 projects (modules) in our repository, counted using the tooling/scm/scripts/gen-repo-candidates.sh
script ( repo-candidates.txt ).
Currently we consider every Maven project for extraction into its own repository. As exception we have:
- tooling/ide, which is a releasable unit by itself and not buildable as individual modules due to toolchain limitations
- tooling/scm, which contains various utility scripts
- tooling/jenkins, which contains the Groovy scripts used to generate the Jenkins jobs
Additionally, we will probably have one aggregator repo used to generate the full view over the git repositories, using repo, gitslave, or another tool that we will settle on.
Several Apache projects (e.g. ServiceMix, Karaf, Camel) moved from Subversion to Git. Is it time for Sling to move also?
see discussion on dev@ and SLING-3987
Can Sling (committers and contributors) benefit from moving to Git?
Pros
- offline support
- cherry picking
- easy to move projects (e.g. between bundles, contrib, whiteboard...)
- ...
Cons
- not eating our own dog food
- slow for big repositories (many modules)
- tagging is global
- ...
What needs to be prepared for moving to Git?
- repository layout
- release process
- ...
Git at Apache
http://wiki.apache.org/general/GitAtApache
http://www.apache.org/dev/git.html
Repository Layout and Projects (Modules)
There are currently over 280 projects (modules) in our repository and only few are reactor/builder (sling
, *reactor
, *builder
) projects:
adapter-annotations
apache-sling-jar-resource-bundle
inplace-integration-test
installing-dependencies-sample
maven-jcrocm-plugin
maven-jspc-plugin
maven-launchpad-plugin
maven-sling-plugin
org.apache.sling.adapter
org.apache.sling.api
org.apache.sling.atom.taglib
org.apache.sling.auth.core
org.apache.sling.auth.form
org.apache.sling.auth.openid
org.apache.sling.auth.selector
org.apache.sling.auth.xing.api
org.apache.sling.auth.xing.login
org.apache.sling.auth.xing.oauth
org.apache.sling.bgservlets
org.apache.sling.bundleresource.impl
org.apache.sling.cassandra
org.apache.sling.commons.cache
org.apache.sling.commons.cache.api
org.apache.sling.commons.cache.container-test
org.apache.sling.commons.cache.ehcache
org.apache.sling.commons.cache.impl
org.apache.sling.commons.cache.infinispan
org.apache.sling.commons.cache.portal
org.apache.sling.commons.classloader
org.apache.sling.commons.compiler
org.apache.sling.commons.fsclassloader
org.apache.sling.commons.html
org.apache.sling.commons.js.nodetypes
org.apache.sling.commons.json
org.apache.sling.commons.log
org.apache.sling.commons.logservice
org.apache.sling.commons.mime
org.apache.sling.commons.osgi
org.apache.sling.commons.scheduler
org.apache.sling.commons.testing
org.apache.sling.commons.threaddump
org.apache.sling.commons.threads
org.apache.sling.crankstart.api
org.apache.sling.crankstart.api.fragment
org.apache.sling.crankstart.core
org.apache.sling.crankstart.launcher
org.apache.sling.crankstart.reactor
org.apache.sling.crankstart.sling.extensions
org.apache.sling.crankstart.test.services
org.apache.sling.datasource
org.apache.sling.discovery.api
org.apache.sling.discovery.impl
org.apache.sling.discovery.standalone
org.apache.sling.discovery.support
org.apache.sling.distribution.api
org.apache.sling.distribution.core
org.apache.sling.distribution.it
org.apache.sling.distribution.sample
org.apache.sling.engine
org.apache.sling.event
org.apache.sling.event.dea
org.apache.sling.extensions.apt.parser
org.apache.sling.extensions.apt.servlet
org.apache.sling.extensions.classloader-leak-detector
org.apache.sling.extensions.dojo
org.apache.sling.extensions.dojo.sling
org.apache.sling.extensions.explorer
org.apache.sling.extensions.gwt.explorer
org.apache.sling.extensions.gwt.sample
org.apache.sling.extensions.gwt.servlet
org.apache.sling.extensions.logback-groovy-fragment
org.apache.sling.extensions.slf4j.mdc
org.apache.sling.extensions.webconsolebranding
org.apache.sling.extensions.webconsolesecurityprovider
org.apache.sling.featureflags
org.apache.sling.fragment.activation
org.apache.sling.fragment.transaction
org.apache.sling.fragment.ws
org.apache.sling.fragment.xml
org.apache.sling.fsresource
org.apache.sling.ftpserver
org.apache.sling.hc.annotations
org.apache.sling.hc.core
org.apache.sling.hc.it
org.apache.sling.hc.junit.bridge
org.apache.sling.hc.reactor
org.apache.sling.hc.samples
org.apache.sling.hc.support
org.apache.sling.hc.webconsole
org.apache.sling.i18n
org.apache.sling.ide.api
org.apache.sling.ide.api-test
org.apache.sling.ide.artifacts
org.apache.sling.ide.eclipse-core
org.apache.sling.ide.eclipse-m2e-test
org.apache.sling.ide.eclipse-m2e-ui
org.apache.sling.ide.eclipse-test
org.apache.sling.ide.eclipse-ui
org.apache.sling.ide.feature
org.apache.sling.ide.impl-resource
org.apache.sling.ide.impl-resource-test
org.apache.sling.ide.impl-vlt
org.apache.sling.ide.impl-vlt-test
org.apache.sling.ide.m2e-feature
org.apache.sling.ide.p2update
org.apache.sling.ide.source-bundle
org.apache.sling.ide.vlt-wrapper
org.apache.sling.installer.console
org.apache.sling.installer.core
org.apache.sling.installer.factory.configuration
org.apache.sling.installer.factory.deploymentpackage
org.apache.sling.installer.factory.subsystems
org.apache.sling.installer.it
org.apache.sling.installer.provider.file
org.apache.sling.installer.provider.jcr
org.apache.sling.javax.activation
org.apache.sling.jcr.api
org.apache.sling.jcr.base
org.apache.sling.jcr.classloader
org.apache.sling.jcr.compiler
org.apache.sling.jcr.contentloader
org.apache.sling.jcr.davex
org.apache.sling.jcr.jackrabbit.accessmanager
org.apache.sling.jcr.jackrabbit.base
org.apache.sling.jcr.jackrabbit.client
org.apache.sling.jcr.jackrabbit.server
org.apache.sling.jcr.jackrabbit.usermanager
org.apache.sling.jcr.jcr-wrapper
org.apache.sling.jcr.oak.server
org.apache.sling.jcr.ocm
org.apache.sling.jcr.prefs
org.apache.sling.jcr.registration
org.apache.sling.jcr.repository.it-jackrabbit-oak
org.apache.sling.jcr.repository.it-resource-versioning
org.apache.sling.jcr.resource
org.apache.sling.jcr.resourcesecurity
org.apache.sling.jcr.webconsole
org.apache.sling.jcr.webdav
org.apache.sling.jcr.workspacepicker
org.apache.sling.jmx.jolokia
org.apache.sling.jmx.provider
org.apache.sling.junit.core
org.apache.sling.junit.healthcheck
org.apache.sling.junit.performance
org.apache.sling.junit.remote
org.apache.sling.junit.scriptable
org.apache.sling.launchpad
org.apache.sling.launchpad.api
org.apache.sling.launchpad.base
org.apache.sling.launchpad.content
org.apache.sling.launchpad.contrib-testing
org.apache.sling.launchpad.installer
org.apache.sling.launchpad.integration-tests
org.apache.sling.launchpad.karaf
org.apache.sling.launchpad.karaf-features
org.apache.sling.launchpad.karaf-integration-tests
org.apache.sling.launchpad.karaf-launchpad-integration-tests
org.apache.sling.launchpad.test-bundles
org.apache.sling.launchpad.test-fragment
org.apache.sling.launchpad.test-services
org.apache.sling.launchpad.test-services-war
org.apache.sling.launchpad.testing
org.apache.sling.launchpad.testing-war
org.apache.sling.mailarchive.james.wrapper
org.apache.sling.mailarchive.reactor
org.apache.sling.mailarchive.server
org.apache.sling.mailarchive.stats
org.apache.sling.mailarchive.ui
org.apache.sling.models.api
org.apache.sling.models.impl
org.apache.sling.models.integration-tests
org.apache.sling.mongodb
org.apache.sling.oak-s3
org.apache.sling.osgi.obr
org.apache.sling.paxexam.util
org.apache.sling.performance.base
org.apache.sling.performance.jcr.resource-2.0.10
org.apache.sling.performance.jcr.resource-2.1.0
org.apache.sling.performance.jcr.resource-2.2.0
org.apache.sling.performance.jcr.resource-2.2.10
org.apache.sling.performance.reactor
org.apache.sling.performance.tests
org.apache.sling.provisioning.model
org.apache.sling.query
org.apache.sling.reqanalyzer
org.apache.sling.resource-editor
org.apache.sling.resource.inventory
org.apache.sling.resourceaccesssecurity
org.apache.sling.resourceaccesssecurity.it
org.apache.sling.resourcecollection
org.apache.sling.resourcemerger
org.apache.sling.resourceresolver
org.apache.sling.rewriter
org.apache.sling.sample.slingshot
org.apache.sling.samples.accessmanager.ui
org.apache.sling.samples.custom-login-form
org.apache.sling.samples.custom-selector-login-form
org.apache.sling.samples.espblog
org.apache.sling.samples.fling
org.apache.sling.samples.frameworkfragment
org.apache.sling.samples.javashell
org.apache.sling.samples.jcr.contentloader
org.apache.sling.samples.path-based.rtp
org.apache.sling.samples.post-servlet-extensions
org.apache.sling.samples.slingbucks
org.apache.sling.samples.urlfilter
org.apache.sling.samples.usermanager.ui
org.apache.sling.samples.webloader.service
org.apache.sling.samples.webloader.ui
org.apache.sling.scripting.api
org.apache.sling.scripting.console
org.apache.sling.scripting.core
org.apache.sling.scripting.freemarker
org.apache.sling.scripting.groovy
org.apache.sling.scripting.java
org.apache.sling.scripting.javascript
org.apache.sling.scripting.jsp
org.apache.sling.scripting.jsp.jstl
org.apache.sling.scripting.jsp.taglib
org.apache.sling.scripting.jst
org.apache.sling.scripting.python
org.apache.sling.scripting.ruby
org.apache.sling.scripting.scala.config
org.apache.sling.scripting.scala.forum
org.apache.sling.scripting.scala.hello-world
org.apache.sling.scripting.scala.script
org.apache.sling.scripting.sightly
org.apache.sling.scripting.sightly.js.provider
org.apache.sling.scripting.sightly.reactor
org.apache.sling.scripting.sightly.repl
org.apache.sling.scripting.sightly.testing
org.apache.sling.scripting.sightly.testing-content
org.apache.sling.scripting.thymeleaf
org.apache.sling.scripting.velocity
org.apache.sling.scripting.xproc
org.apache.sling.security
org.apache.sling.serviceusermapper
org.apache.sling.servlets.compat
org.apache.sling.servlets.get
org.apache.sling.servlets.post
org.apache.sling.servlets.resolver
org.apache.sling.settings
org.apache.sling.slingstart
org.apache.sling.startupfilter
org.apache.sling.startupfilter.disabler
org.apache.sling.superimposing
org.apache.sling.tenant
org.apache.sling.testing.jcr-mock
org.apache.sling.testing.osgi-mock
org.apache.sling.testing.reactor
org.apache.sling.testing.resourceresolver-mock
org.apache.sling.testing.samples.failingtests
org.apache.sling.testing.samples.integrationtests
org.apache.sling.testing.samples.sampletests
org.apache.sling.testing.sling-mock
org.apache.sling.testing.sling-mock-jackrabbit
org.apache.sling.testing.tools
org.apache.sling.tooling.support.install
org.apache.sling.urlrewriter
org.apache.sling.validation.api
org.apache.sling.validation.core
org.apache.sling.validation.examples
org.apache.sling.validation.it-http
org.apache.sling.validation.reactor
org.apache.sling.validation.test-services
org.apache.sling.xss
reactor
sample-war
sling
sling-archetype-parent
sling-builder
sling-bundle-archetype
sling-contrib-builder
sling-initial-content-archetype
sling-jcrinstall-bundle-archetype
sling-lauchpad
sling-launchpad-karaf-builder
sling-launchpad-standalone-archetype
sling-launchpad-webapp-archetype
sling-samples-builder
sling-servlet-archetype
sling-taglib-archetype
slingstart-maven-plugin
Idea
- one Git repository per project/module (we should keep it that simple - is launchpad really different, a release unit, Justin Edelson?)
- grouping repositories with Google repo, e.g.
core
(bundles
),launchpad
,testing
,samples
,tooling
as needed - using
artifactId
as repository name for artifacts (bundles, plugins, jars, ...) and simple names for grouping (builder/reactor) repositories - using Google repo allows developers to set up their own grouping repositories - even non Sling committers
Grouping repositories (projects/modules)
There are some tools for grouping repositories (projects/modules):
Some posts comparing different tools:
- http://blogs.atlassian.com/2013/05/alternatives-to-git-submodule-git-subtree/
- https://codingkilledthecat.wordpress.com/2012/04/28/why-your-company-shouldnt-use-git-submodules/
Pros and Cons
[TODO]
...
References
dev@discussions:
- [jira] [Created] (SLING-3987) move from Subversion to Git
- moving sling to git
- [git] Which git system do we want to use?
- [git] Please test sample migrated git projects and repo setup
- [git] Whiteboard migration