...
...
Minimum Requirements
See the Cluster Requirements.
...
| Processors per node | - small = 2 cores
- medium = 4 cores
- large = 8 cores or more
|
| Memory per node | - small = 16 GB of RAM
- medium = 64 GB or RAM
- large = 128 GB of RAM
|
| Number of nodes and concurrency | - 2 small nodes = 4 concurrent queries
- 2 medium nodes = 64 concurrent queries
- 2 large nodes = 256 concurrent queries
|
| HBase memory per node | This setting will vary depending on the type of queries and concurrency level. Use these as starting values:- small: Region Server Java Heap Size = 2 GB
- medium: Region Server Java Heap Size = 8 GB
- large: Region Server Java Heap Size = 16 GB
|
HBase Configuration and Fine Tuning
Installer Modifications to HBase
...
NOTE: To optimize the performance of a Trafodion system, we recommend that you enable HBase balancing during a maintenance window. For recommendations, see Maintenance Window.
| Anchor |
|---|
| HadoopDistroConfig |
|---|
| HadoopDistroConfig |
|---|
|
Hadoop Distribution Configuration and Fine Tuning
It is recommended that you modify these HDFS heap size settings as described below.
| HDFS Property | Recommended Setting | Notes |
|---|
| DataNode Java Heap Size | 2 GiB | Use this setting for a large configuration. |
| NameNode Java Heap Size | 2 GiB | Use this setting for a large configuration. |
| Secondary NameNode Java Heap Size | 2 GiB | Use this setting for a large configuration. |
Trafodion Configuration and Fine Tuning
| Anchor |
|---|
| TrafConfigConnectivity |
|---|
| TrafConfigConnectivity |
|---|
|
Connectivity
In the Trafodion software, the heap size for Database Connectivity Services (DCS) is set by default to 2GB. However, in our testing we found that limiting the DcsMaster to 64MB of heap size resulted in better throughput.
...
For ODBC connections on Linux workstations, we found that using at least version 2.3.x of the third-party driver manager, unixODBC, provided the best throughput. For more information about how to install the Trafodion ODBC Driver for Linux, see the Trafodion Client Installation Guide (pdf, 1.12 MB).
| Anchor |
|---|
| TrafConfigTableMaint |
|---|
| TrafConfigTableMaint |
|---|
|
Table Maintenance
It is important to maintain current statistics for your Trafodion tables. This allows the compiler to generate optimal Trafodion execution plans in a performant manner. The update statistics SQL command will generate up-to-date statistics for a table. To perform an update statistics command, we recommend a basic query.
...
[trafodion@n1~]$ trafci
Welcome to Trafodion Command Interface
Copyright(C) 2013-2014 Hewlett-Packard Development Company, L.P.
Host Name/IP Address: localhost:37800
User Name: zz
Connected to Trafodion
SQL>UPDATE STATISTICS FOR TABLE TRAFODION.SEABASE.TABLE1 ON EVERY COLUMN SAMPLE;
--- SQL operation complete.
SQL>exit;
| Anchor |
|---|
| TrafConfigDataLoad |
|---|
| TrafConfigDataLoad |
|---|
|
Data Load
To maximize the performance of loading data into Trafodion tables, we recommend using the UPSERT USING LOAD syntax.
...
For more information, see Data Loading.
| Anchor |
|---|
| TrafConfigJVM |
|---|
| TrafConfigJVM |
|---|
|
JVM Heap Size
The default maximum heap size is 2 GB. This setting results in a large allocation of resident memory space. We have reduced this value to 512 MB for OLTP workloads to reduce the default memory consumption of mxosrvr and tdd_arkcmp processes.
| Anchor |
|---|
| TrafConfigTablePart |
|---|
| TrafConfigTablePart |
|---|
|
Table Partitioning and Salting
We choose to load all user tables with a pre-salting equal to the number of region servers on the system. For example: create table … salt using <n> partitions where <n> is the number of regions servers on the system.
| Anchor |
|---|
| TrafConfigOLTPPerf |
|---|
| TrafConfigOLTPPerf |
|---|
|
OLTP Performance Improvements
For improved OLTP performance, HBase and the Trafodion DTM should be configured to match the same level of concurrency as DCS.
- In the HBase configuration: hbase.regionserver.handler.count=128
- In
$SQ_HOME/etc/ms.env: TM_JAVA_THREAD_POOL_SIZE=128
Maintenance Window
HBase, in its normal operation, performs memstore flushes and compactions per region. Region splits and balancing also occur to help optimize HBase load and performance. However, the frequency and duration of those activities can adversely affect overall Trafodion transaction execution.
...
