You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

This document describes the architecture of the current CI system hosted at http://jenkins.mxnet-ci.amazon-ml.com/

 

State persistence and versioning

Analysis

To preserve the state of the Jenkins Master, the various files in the Jenkins configuration directory at /var/lib/jenkins have to be considered. In general, these files can be categorized as follows:

  • Config: Configuration files
  • Cache: Temporary cache files to improve performance
  • State: Related to the current and past execution
  • Secrets: Special configuration files which have to be handled with pre-caution (aka must not be stored on a versioning system but rather in a secure storage like KMS).

While Config and Secrets are a crucial part of the CI system and thus require detailed backups and versioning, the Cache can be stored in a non-persistent way. It would be desirable to have the State persisted as this allows to have a seamless experience in between deployments. Due to the fact that the State changes very frequently, no explicit versioning is required.

The analysis and categorization of the existing files is as follows:

caches: Cache
fingerprints: State
HOOK.groovy.d: Config
jobs: Config
	incubator-mxnet: Config
		branches: State
		indexing: State
		config.xml: Config
		state.xml: Config
    ... 
logs: 
nodes
plugins
secrets
updates
userContent
users
workflow-libs
workspace

com.cloudbees.jenkins.plugins.bitbucket.endpoints.BitbucketEndpointConfiguration.xml
com.orctom.jenkins.plugin.buildtimestamp.BuildTimestampWrapper.xml
config.xml
credentials.xml
github-plugin-configuration.xml
hudson.model.UpdateCenter.xml
hudson.plugins.ansicolor.AnsiColorBuildWrapper.xml
hudson.plugins.build_timeout.operations.BuildStepOperation.xml
hudson.plugins.emailext.ExtendedEmailPublisher.xml
hudson.plugins.git.GitSCM.xml
hudson.plugins.git.GitTool.xml
hudson.plugins.jira.JiraProjectProperty.xml
hudson.plugins.timestamper.TimestamperConfig.xml
hudson.tasks.Mailer.xml
hudson.tasks.Shell.xml
hudson.triggers.SCMTrigger.xml
identity.key.enc
javaposse.jobdsl.plugin.GlobalJobDslSecurityConfiguration.xml
jenkins.CLI.xml
jenkins.install.InstallUtil.lastExecVersion
jenkins.install.UpgradeWizard.state
jenkins.metrics.api.MetricsAccessKey.xml
jenkins.model.ArtifactManagerConfiguration.xml
jenkins.model.DownloadSettings.xml
jenkins.model.JenkinsLocationConfiguration.xml
jenkins.security.QueueItemAuthenticatorConfiguration.xml
jenkins.security.UpdateSiteWarningsConfiguration.xml
nodeMonitors.xml
org.jenkinsci.plugins.ghprb.GhprbTrigger.xml
org.jenkinsci.plugins.pipeline.modeldefinition.config.GlobalConfig.xml
org.jenkinsci.plugins.resourcedisposer.AsyncResourceDisposer.xml
org.jenkinsci.plugins.workflow.flow.FlowExecutionList.xml
org.jenkinsci.plugins.workflow.libs.GlobalLibraries.xml
org.jvnet.hudson.plugins.SSHBuildWrapper.xml
queue.xml.bak
scriptApproval.xml
secret.key
secret.key.not-so-secret

Solution

asdasd

  • No labels