You are viewing an old version of this page. View the current version.
Compare with Current
View Page History
« Previous
Version 14
Next »
This document describes the architecture of the current CI system hosted at http://jenkins.mxnet-ci.amazon-ml.com/
State persistence and versioning
Analysis
To preserve the state of the Jenkins Master, the various files in the Jenkins configuration directory at /var/lib/jenkins have to be considered. In general, these files can be categorized as follows:
- Config: Configuration files
- Cache: Temporary cache files to improve performance
- State: Related to the current and past execution
- Secrets: Special configuration files which have to be handled with pre-caution (aka must not be stored on a versioning system but rather in a secure storage like KMS).
While Config and Secrets are a crucial part of the CI system and thus require detailed backups and versioning, the Cache can be stored in a non-persistent way. It would be desirable to have the State persisted as this allows to have a seamless experience in between deployments. Due to the fact that the State changes very frequently, no explicit versioning is required.
The analysis and categorization of the existing files is as follows:
Spoiler: File analysis and categorization
caches: Cache
fingerprints: State
HOOK.groovy.d: Config
jobs: Config
incubator-mxnet: Config
branches: State
indexing: State
config.xml: Config
state.xml: Config
...
logs: State
nodes: Config
plugins: Config
secrets: Secrets
updates: Config
userContent: Config
users: Config or State AND Secret ?????
workflow-libs: Config
workspace: State
com.cloudbees.jenkins.plugins.bitbucket.endpoints.BitbucketEndpointConfiguration.xml: Config
com.orctom.jenkins.plugin.buildtimestamp.BuildTimestampWrapper.xml: Config
config.xml: Config AND Secret
credentials.xml: Secret
github-plugin-configuration.xml: Config
hudson.model.UpdateCenter.xml: Config
hudson.plugins.ansicolor.AnsiColorBuildWrapper.xml: Config
hudson.plugins.build_timeout.operations.BuildStepOperation.xml: Config
hudson.plugins.emailext.ExtendedEmailPublisher.xml: Config
hudson.plugins.git.GitSCM.xml: Config
hudson.plugins.git.GitTool.xml: Config
hudson.plugins.jira.JiraProjectProperty.xml: Config
hudson.plugins.timestamper.TimestamperConfig.xml: Config
hudson.tasks.Mailer.xml: Config
hudson.tasks.Shell.xml: Config
hudson.triggers.SCMTrigger.xml: Config
identity.key.enc: Secret
javaposse.jobdsl.plugin.GlobalJobDslSecurityConfiguration.xml: Config
jenkins.CLI.xml: Config
jenkins.install.InstallUtil.lastExecVersion: Config
jenkins.install.UpgradeWizard.state: Config
jenkins.metrics.api.MetricsAccessKey.xml: Config
jenkins.model.ArtifactManagerConfiguration.xml: Config
jenkins.model.DownloadSettings.xml: Config
jenkins.model.JenkinsLocationConfiguration.xml: Config
jenkins.security.QueueItemAuthenticatorConfiguration.xml: Config
jenkins.security.UpdateSiteWarningsConfiguration.xml: Config
nodeMonitors.xml: Config
org.jenkinsci.plugins.ghprb.GhprbTrigger.xml: Config and Secret
org.jenkinsci.plugins.pipeline.modeldefinition.config.GlobalConfig.xml: Config
org.jenkinsci.plugins.resourcedisposer.AsyncResourceDisposer.xml: Config
org.jenkinsci.plugins.workflow.flow.FlowExecutionList.xml: State
org.jenkinsci.plugins.workflow.libs.GlobalLibraries.xml: Config
org.jvnet.hudson.plugins.SSHBuildWrapper.xml: Config
queue.xml.bak: State
scriptApproval.xml: Config
secret.key: Secret
secret.key.not-so-secret: Secret
Solution
asdasd