...
Such properties can be defined in .json format in the properties folder of the stack.
+
Stack features
Stacks can support different features depending on their version, for example: upgrade support, NFS support, support for specific new components (such as Ranger, Phoenix )…
...
Code Block | ||||
---|---|---|---|---|
| ||||
"stack_features": [
{
"name": "snappy",
"description": "Snappy compressor/decompressor support",
"min_version": "2.0.0.0",
"max_version": "2.2.0.0"
},
{
"name": "lzo",
"description": "LZO libraries support",
"min_version": "2.2.1.0"
},
{
"name": "express_upgrade",
"description": "Express upgrade support",
"min_version": "2.1.0.0"
},
{
"name": "rolling_upgrade",
"description": "Rolling upgrade support",
"min_version": "2.2.0.0"
}
]
} |
where min_version/max_version are optional constraints.
Feature constants, matching features names, such has ROLLING_UPGRADE = "rolling_upgrade" has been added to a new StackFeature class in resource_management/libraries/functions/constants.py
Code Block | ||||
---|---|---|---|---|
| ||||
class StackFeature:
"""
Stack Feature supported
"""
SNAPPY = "snappy"
LZO = "lzo"
EXPRESS_UPGRADE = "express_upgrade"
ROLLING_UPGRADE = "rolling_upgrade" |
Additionally, corresponding helper functions has been introduced in resource_management/libraries/functions/stack_fetaures.py to parse the .json file content and called from service code to check if the stack supports the specific feature.
This is an example where the new stack featurization design is used in service code:
Code Block | ||||
---|---|---|---|---|
| ||||
if params.version and check_stack_feature(StackFeature.ROLLING_UPGRADE, params.version):
conf_select.select(params.stack_name, "hive", params.version)
stack_select.select("hive-server2", params.version) |
Stack Tools
Similar to stack features, stack-selector and conf-selector tools are now stack-driven instead of hardcoding hdp-select and conf-select. They are defined in stack_tools.json file under /HDP/2.0.6/properties
And declared as part of the HDP stack configurations as a new property on /HDP/2.0.6/configuration/cluster-env.xml
Code Block | ||||
---|---|---|---|---|
| ||||
<!-- Define stack_tools property in the base stack. DO NOT override this property for each stack version -->
<property>
<name>stack_tools</name>
<value/>
<description>Stack specific tools</description>
<property-type>VALUE_FROM_PROPERTY_FILE</property-type>
<value-attributes>
<property-file-name>stack_tools.json</property-file-name>
<property-file-type>json</property-file-type>
<read-only>true</read-only>
<overridable>false</overridable>
<visible>false</visible>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property> |
Corresponding helper functions have been added in resource_management/libraries/functions/stack_tools.py. These helper functions are used to remove hardcodings in resource_management library.