DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
Page History
Introduction
CloudStack DB upgrade takes care of updating the database schema and updating data (if needed) between the releases. The upgrade is initiated by the cloud-management process start - if the version of the software installed is greater than the data base version, the upgrade is being performed.
DB upgrade code structure
- When management server starts up, we perform number of system integrity checks. All the checker classes inherit from SystemIntegrityChecker interface, and have to be listed in components.xml configuration file (last 2 checks are DB upgrade related checks): <system-integrity-checker>
<checker
...
- name="ManagementServerNode"/>
...
...
- <checker
...
- name="EncryptionSecretKeyChecker"/>
...
...
- <checker
...
- name="DatabaseIntegrityChecker"/>
...
...
- <checker
...
- name="DatabaseUpgradeChecker"/>
...
...
- </system-integrity-checker>The
...
- checkers
...
- are
...
- executed
...
- in
...
- the
...
- order
...
- they
...
- are
...
- defined.
...
DataBaseIntegrityChecker verifies if the DB ready for the upgrade. When this check fails, user has to perform some manual modifications in order to make it work. As an example, in 2.1.x
...
we
...
had
...
a
...
bug
...
when
...
single
...
Local
...
Storage
...
was
...
associated
...
with
...
more
...
than
...
one
...
Hypervisors
...
in
...
the
...
DB,
...
so
...
user
...
had
...
to
...
remove one of the references.
...
We
...
couldn't
...
do
...
it
...
on
...
behalf
...
of
...
him,
...
because
...
we
...
had
...
no
...
idea
...
which
...
reference
...
is
...
the
...
correct
...
one.
...
DataBaseUpgradeChecker does the actual upgrade. In 3.0.x
...
code
...
2
...
classes
...
implement
...
this
...
interface
...
- DataBaseUpgradeChecker and PremiumDataBaseUpgradeChecker.
...
It's
...
left
...
from
...
the
...
times
...
when
...
we
...
had
...
Premium
...
and
...
Open
...
Source
...
versions
...
of
...
cloudStack
...
software,
...
and
...
DB/DB
...
upgrade
...
procedures
...
varied
...
based
...
on
...
the
...
version.
...
We
...
are
...
planning
...
to
...
merge
...
these
...
2
...
checkers
...
in
...
the
...
near
...
future.
...
- DataBaseUpgradeChecker
...
- determines
...
- the
...
- current
...
- version
...
- of
...
- the
...
- software
...
- from
...
- the
...
- RPM
...
- package,
...
- and
...
- compares
...
- it
...
- with
...
- the
...
- latest
...
- DB
...
- version
...
- taken
...
- from
...
- the
...
- last
...
- row in cloud.version
...
- table.
...
- If
...
- the
...
- DB
...
- version
...
- is
...
- less
...
- than
...
- the
...
- software
...
- version,
...
- we
...
- decide
...
- which
...
- set
...
- of
...
- upgrade
...
- files
...
- should
...
- be
...
- executed
...
- -
...
- get
...
- the
...
- set
...
- from
...
- _
...
- upgradeMap
...
- table.
- For each file defined in upgrade set, DataBaseUpgradeChecker 1) DBUpgrade.getPrepareScripts()
...
- and
...
- run
...
- the
...
- scripts
...
- with
...
- runScript()
...
- 2)
...
- DBUpgrade.performDataMigration().
...
- Execution
...
- order
...
- is
...
- the
...
- same
...
- as
...
- the
...
- files
...
- appear
...
- in
...
- upgrade
...
- set.
...
- After
...
- prepareScripts
...
- and
...
- dataMigration
...
- is
...
- finished,
...
- DataBaseUpgradeChecker
...
- executes
...
- the
...
- cleanup
...
- part
...
- for
...
- all
...
- the
...
- files
...
- from
...
- upgrade
...
- set
...
- -
...
- DBUpgrade.getCleanupScripts()
...
- and
...
- run
...
- them
...
- with
...
- runScript().
...
- Cleanup
...
- scripts
...
- are
...
- very
...
- useful
...
- in
...
- case
...
- when
...
- some
...
- data
...
- has
...
- to
...
- be
...
- dropped,
...
- but
...
- only
...
- at
...
- the
...
- end
...
- of
...
- the
...
- upgrade
...
- as
...
- we
...
- rely
...
- on
...
- this
...
- data
...
- while
...
- performing
...
- the
...
- "Data
...
- Migration"
...
- part.
...
Adding new upgrade path
Lets assume we already released 3.0.2
...
version
...
of
...
the
...
product,
...
and
...
started
...
development
...
for
...
3.0.3.
...
If
...
anything
...
from
...
below
...
is
...
modified
...
in
...
3.0.3
...
release,
...
we
...
need
...
to
...
provide
...
the
...
DB
...
upgrade:
...
- new
...
- DB
...
- table
...
- is
...
- added
...
- field
...
- was
...
- added/removed
...
- from/to
...
- existing
...
- table
...
- indexes
...
- were
...
- added/removed
...
- from/to
...
- existing
...
- tables
...
- changes
...
- to
...
- the
...
- data
...
- were
...
- made
...
- -
...
- new
...
- config
...
- variable
...
- is
...
- inserted
...
- to the cloud.configuration
...
- table,
...
- the
...
- field
...
- variable
...
- was
...
- modified
...
- (say,
...
- we
...
- changed
...
- Running
...
- state
...
- of
...
- the
...
- vm
...
- to
...
- say
...
- Alive
...
- instead)
...
1)
...
Add
...
Upgrade302to303()
...
implementing DbUpgrade
2)
...
Add
...
prepare
...
and
...
cleanup
...
scripts
...
(if
...
needed)
...
to
...
setup/db/db
...
directory.
...
The
...
naming
...
convention
...
is
...
"schema-<versiontoupgradefrom>to<versinotoupgradeto>.sql"
...
and "schema-<versiontoupgradefrom>to<versinotoupgradeto>-cleanup.sql":
...
/setup/db/db/schema-302to303.sql
...
/setup/db/db/schema-302to303-cleanup.sql
...
In
...
schema-302to303.sql
...
specify
...
all
...
the
...
upgrade
...
steps
...
that
...
can
...
be
...
made
...
without
...
java
...
code
...
help
...
-
...
dropping
...
the
...
fields,
...
changing
...
the
...
field
...
name,
...
etc.
...
In
...
schema-302to303-cleanup.sql
...
specify
...
all
...
cleanup
...
steps.
...
It
...
will
...
be
...
executed
...
only
...
after
...
prepared
...
scripts
...
and
...
data
...
migration
...
is
...
executed.
...
We
...
rarely
...
need
...
cleanup
...
scripts,
...
only
...
when
...
some
...
field/value
...
is
...
required
...
during
...
the
...
data
...
migration
...
and
...
has
...
to
...
be
...
dropped
...
later
...
on.
...
3)
...
Add
...
upgrade
...
path
...
from
...
3.0.2
...
to
...
_upgradeMap in DataBaseUpgradeChecker; example:
| No Format |
|---|
*upgradeMap* in *DataBaseUpgradeChecker*; example:_upgradeMap.put("3.0.2", new DbUpgrade[] { new Upgrade302to303() } ); |
4)
...
Append
...
"new
...
Upgrade302to303()"
...
to
...
all
...
entries
...
in
...
_upgradeMap:
| No Format |
|---|
\_upgradeMap.put("3.0.1", new DbUpgrade\[\] { new Upgrade301to302(), new Upgrade302to303() } ); |
5)
...
In
...
Upgrade302to303()
...
- Set
...
- the
...
- version
...
- to
...
- upgrade
...
- from
...
- and
...
- the
...
- version
...
- to
...
- upgrade
...
- to
...
- in method:
| No Format |
|---|
method: public String\[\] getUpgradableVersionRange() { return new String\[\] { "3.0.2", "3.0.3" } ; } * |
- Set
...
- the
...
- version
...
- to
...
- upgrade
...
- to
...
- in
...
- method:
| No Format |
|---|
public String getUpgradedVersion() { { return "3.0.3"; } |
pecify all the Prepare scripts in:
No Format public File[] getPrepareScripts() { throw new CloudRuntimeException("Unable to find db/schema-301to302.sql"); } return new File[] { new File(script) } ; }* Specify
...
the
...
cleanup scripts in method:
...
public File[]
...
getCleanupScripts() { String script = Script.findScript("", "db/schema-301to302-cleanup.sql"); if (script == null) { throw new CloudRuntimeException("Unable to find db/schema-301to302-cleanup.sql"); } return new File[] { new File(script) } ; " macrohasbody="false" wikihasprecedingnewline="false" wikihastrailingnewline="false" > { String script = Script.findScript("", "db/schema-301to302.sql"); if (script == null) { throw new CloudRuntimeException("Unable to find db/schema-301to302.sql");
...
} return new File[]
...
{ new File(script) } ; }* Specify the cleanup scripts in method: public File[] getCleanupScripts() { String script = Script.findScript("", "db/schema-301to302-cleanup.sql"); if (script == null) {
...
throw new CloudRuntimeException("Unable to find db/schema-301to302-cleanup.sql");
...
} return new File[]
...
{ new File(script) } ;
...
}
...
If
...
any
...
data
...
migration
...
is
...
needed,
...
code
...
it
...
as
...
a
...
part
...
of
...
performDataMigration().
...
6)
...
And
...
the
...
last
...
step
...
-
...
always
...
add
...
the
...
and
...
execute
...
UnitTest
...
for
...
the
...
db
...
upgrade
...
path.
...
You
...
can
...
look
...
at
...
AdvanceZone217To224UpgradeTest to get the idea.
Hints
- When you want to drop the field/key that existed in N-2 version, but was missing in N-1 version, you have to handle both scenarios in N-1->N
...
- upgrade.
...
- Do
...
- it
...
- in
...
- the
...
- Java
...
- code,
...
- performDataMigration()
...
- method
...
- using
...
- these
...
- calls
...
- from
...
- DbUpgradeUtils
...
- helper
...
- class:
...
...
- -
...
- dropKeysIfExist
...
...
- -
...
- dropPrimaryKeyIfExists
...
...
- -
...
- dropTableColumnsIfExist*
...
- Always
...
- close
...
- ResultSet
...
- and
...
- PreparedStatements
...
- in
...
- finally()
...
- block
...
- during
...
- the
...
- DataMigration.
...
- It
...
- helps
...
- to
...
- eliminate
...
- memory
...
- leaks.
...
If
...
you
...
want
...
to
...
suppress
...
errors
...
in
...
INSERT
...
(be
...
careful
...
with
...
this
...
though),
...
use
...
INSERT
...
IGNORE
...
as
...
opposed
...
to
...
INSERT.
...
We
...
mostly
...
use
...
it
...
when
...
have
...
to
...
insert
...
global
...
config
...
variable
...
missing
...
in
...
just
...
some
...
of
...
the
...
previous
...
releases:
No Format INSERT IGNORE INTO `cloud`.`configuration` VALUES ('Advanced', 'DEFAULT', 'management-server', 'direct.agent.load.size', '16', 'The number of direct agents to load each time');
...
- Never modify the upgrade path leading to the version that is already released, unless its a blocker bug and the customer needs a patch. Example: 4.1 was released, and there is a bug in 4.0->4.1 upgrade path. The fix should be made in 4.1->4.2 patch, never in 4.0->4.1
Known issues
- No rollback is supported. We always insist on saving the DB dump before running the upgrade so in case something goes wrong, the Admin can do manual rollback. Steps to do the manual rollback:
- Get the patch with the fix for the failed upgrade
- Drop the partially updated database
- Apply previously saved DB dump
- Run the upgrade once again
Overview
Content Tools