This section describes the following:

• Prerequisite Software
• Package Contents
• Ambari SCOM Server Installation
  • Manual Installation
    • Configuring SQL Server
    • Configuring Hadoop Metrics Sink
    • Installing and Configuring Ambari SCOM Server
  • MSI Installation
    • Configuring SQL Server
    • Running the MSI Installer
• Installing Ambari SCOM Management Pack
  • Import the Management Pack
  • Create Run As Account
  • Configure Management Pack

Prerequisite Software

Setting up Ambari SCOM assumes the following prerequisite software:

• Ambari SCOM 1.0
  • Apache Hadoop 1.x cluster (HDFS and MapReduce) ¹
• Ambari SCOM 2.0
  • Apache Hadoop 2.x cluster (HDFS and YARN/MapReduce) ²
• JDK 1.7
• Microsoft SQL Server 2012
• Microsoft JDBC Driver 4.0 for SQL Server ³
• Microsoft System Center Operations Manager (SCOM) 2012 SP1 or later
• System Center Monitoring Agent installed on Watcher Node ⁴

¹ Ambari SCOM 1.0 has been tested with a Hadoop cluster based on Hortonworks Data Platform 1.3 for Windows ("HDP 1.3 for Windows")

² Ambari SCOM 2.0 has been tested with a Hadoop cluster based on Hortonworks Data Platform 2.1 for Windows ("HDP 2.1 for Windows")

³ Obtain the Microsoft JDBC Driver 4.0 for SQL Server JAR file (sqljdbc4.jar) at http://technet.microsoft.com/en-us/library/ms378749.aspx

⁴ See Microsoft TechNet topic for Managing Discovery and Agents. Minimum Agent requirements .NET 4 and PowerShell 2.0 + 3.0

Package Contents

├─ ambari-scom-version.zip
├── README.md
├── server.zip
├── metrics-sink.zip
├── mp.zip
└── ambari-scom.msi

Ambari SCOM Server Installation

The following methods are available for installing Ambari SCOM Server:

• Manual Installation - This installation method requires you to configure the SQL Server database, setup the Ambari SCOM Server and configure the Hadoop Metrics Sink. This provides the most flexible install option based on your environment. See Manual Installation for more information.

• MSI Installation - This installation method installs the Ambari SCOM Server and configures the Hadoop Metrics Sink on all hosts in the cluster automatically using an MSI Installer. After launching the MSI, you provide information about your SQL Server database and the cluster for the installer to handle configuration. See MSI Installation for more information.

Manual Installation

Configuring SQL Server

1. Configure an existing SQL Server instance for "mixed mode" authentication.
2. Confirm SQL Server is installed with TCP/IP active and enabled. (default port: 1433)
3. Create a user and password. Remember this user and password as this will be the account used by the Hadoop metrics interface for capturing metrics. (default user: sa)
4. Extract the contents of the metrics-sink.zip package to obtain the DDL script.
5. Create the Ambari SCOM database schema by running the Hadoop-Metrics-SQLServer-CREATE.ddl script.

Configuring Hadoop Metrics Sink

Preparing the Metrics Sink

1. Extract the contents of the metrics-sink.zip package to obtain the metrics-sink-version.jar file.
2. Obtain the Microsoft JDBC Driver 4.0 for SQL Server sqljdbc4.jar file (see Prerequisite Software).
3. Copy sqljdbc4.jar and metrics-sink-version.jar to each host in the cluster. For example, copy to C:\Ambari\metrics-sink-version.jar and C:\Ambari\sqljdbc4.jar
   on each host.

Setup Hadoop Metrics2 Interface

1. On each host in the cluster, setup the Hadoop metrics2 interface to use the SQLServerSink.

Edit the hadoop-metrics2.properties file (located in the {C:\hadoop\install\dir}\bin folder of each host in the cluster):

*.sink.sql.class=org.apache.hadoop.metrics2.sink.SqlServerSink

namenode.sink.sql.databaseUrl=jdbc:sqlserver://[server]:[port];databaseName=HadoopMetrics;user=[user];password=[password]
datanode.sink.sql.databaseUrl=jdbc:sqlserver://[server]:[port];databaseName=HadoopMetrics;user=[user];password=[password]
jobtracker.sink.sql.databaseUrl=jdbc:sqlserver://[server]:[port];databaseName=HadoopMetrics;user=[user];password=[password]
tasktracker.sink.sql.databaseUrl=jdbc:sqlserver://[server]:[port];databaseName=HadoopMetrics;user=[user];password=[password]
maptask.sink.sql.databaseUrl=jdbc:sqlserver://[server]:[port];databaseName=HadoopMetrics;user=[user];password=[password]
reducetask.sink.sql.databaseUrl=jdbc:sqlserver://[server]:[port];databaseName=HadoopMetrics;user=[user];password=[password]

1. Update the Java classpath for each Hadoop service to include the metrics-sink-version.jar and sqljdbc4.jar files.
   
   

   1. Example: Updating the Java classpath for HDP for Windows clusters
      
      The service.xml files will be located in the {C:\hadoop\install\dir}\bin folder of each host in the cluster. The Java classpath is specified for each service in the <arguments> element of the service.xml file.

      For example, to update the Java classpath for the NameNode component, edit the C:\hadoop\bin\namenode.xml file.

      ...
      <arguments>... -classpath ...;C:\Ambari\metrics-sink-1.5.1.2.0.0.0-673.jar;C:\Ambari\sqljdbc4.jar ...</arguments>
      ...
      

2. Restart Hadoop for these changes to take affect.

Verify Metrics Collection

1. Confirm metrics are being captured in the SQL Server database by querying the MetricRecord table:

   select * from HadoopMetrics.dbo.MetricRecord
   

   In the above SQL statement, HadoopMetrics is the database name.

Installing and Configuring Ambari SCOM Server

Running the Server

1. Designate a machine in the cluster to run the Ambari SCOM Server.
2. Extract the contents of the server.zip package to obtain the Ambari SCOM Server packages.
   ├── ambari-scom-server-version-conf.zip
   ├── ambari-scom-server-version-lib.zip
   └── ambari-scom-server-version.jar
3. Extract the contents of the ambari-scom-server-version-lib.zip package to obtain the Ambari SCOM dependencies.
4. Extract the contents of the ambari-scom-server-version-conf.zip package to obtain the Ambari SCOM configuration files.

5. From the configuration files, edit the ambari.properties file:

   scom.sink.db.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
   scom.sink.db.url=jdbc:sqlserver://[server]:[port];databaseName=HadoopMetrics;user=[user];password=[password]
   

   Where:

   • server = the SQL Server hostname
   • port = the SQL Server port (for example, 1433)
   • user = the SQL Server user (for example, sa)
   • password = the SQL Server password (for example, BigData1)

6. Run the org.apache.ambari.scom.AmbariServer class from the Java command line to start the Ambari SCOM Server.

   Be sure to include the following in the classpath:

   • ambari-scom-server-version.jar file
   • configuration folder containing the Ambari SCOM configuration files
   • lib folder containing the Ambari SCOM dependencies
   • folder containing the clusterproperties.txt file from the Hadoop install. For example, c:\hadoop\install\dir
   • sqljdbc4.jar SQLServer JDBC Driver file (see Prerequisite Software)

   For example:

   java -server -XX:NewRatio=3 -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit -XX:CMSInitiatingOccupancyFraction=60 -Xms512m -Xmx2048m -cp "c:\ambari-scom\server\conf;c:\ambari-scom\server\lib\*;c:\jdbc\sqljdbc4.jar;c:\hadoop\install\dir;c:\ambari-scom\server\ambari-scom-server-1.5.1.2.0.0.0-673.jar" org.apache.ambari.scom.AmbariServer
   

   In the above command, be sure to replace the Ambari SCOM version in the ambari-scom-server-version.jar and replace c:\hadoop\install\dir with the folder containing the clusterproperties.txt file.

Verify the Server API

1. From a browser access the API

   http://[ambari-scom-server]:8080/api/v1/clusters
   

2. Verify that metrics are being reported.

   http://[ambari-scom-server]:8080/api/v1/clusters/ambari/services/HDFS/components/NAMENODE
   

MSI Installation

Configuring SQL Server

1. Configure an existing SQL Server instance for "mixed mode" authentication.
2. Confirm SQL Server is installed with TCP/IP active and enabled. (default port: 1433)
3. Create a user and password. (default user: sa)

Running the MSI Installer

1. Designate a machine in the cluster to run the Ambari SCOM Server.
2. Extract the contents of the server.zip package to obtain the Ambari SCOM Server packages.
3. Run the ambari-scom.msi installer. The "Ambari SCOM Setup" dialog appears:
   

4. Provide the following information:

   Field	Description
   Ambari SCOM package directory	The directory where the installer will place the Ambari SCOM Server packages. For example: C:\Ambari
   SQL Server hostname	The hostname of the SQL Server instance for Ambari SCOM Server to use to store Hadoop metrics.
   SQL Server port	The port of the SQL Server instance.
   SQL Server login	The login username.
   SQL Server password	The login password
   Path to SQL Server JDBC Driver (sqljdbc4.jar)	The path to the JDBC Driver JAR file.
   Path to the cluster layout file (clusterproperties.txt)	The path to the cluster layout properties file.

5. You can optionally select to Start Services
6. Click Install
7. After completion, links are created on the desktop to "Start Ambari SCOM Server", "Browse Ambari API" and "Browse Ambari API Metrics". After starting the Ambari SCOM Server, browse the API and Metrics to confirm the server is working properly.

Installing Ambari SCOM Management Pack

Import the Management Pack

Perform the following to import the Ambari SCOM Management Pack into System Center Operations Manager.

1. Extract the contents of the mp.zip package to obtain the Ambari SCOM management pack (.mpb) files.
2. Ensure Windows Server 2012 running SCOM with SQL Server (full text search).
3. Open System Center Operations Manager.
4. Go to Administration -> Management Packs.
5. From the Tasks panel, select Import Management Packs...
6. In the Import Management Packs dialog, select Add -> Add from disk...
7. You are prompted to search the Online Catalog. Click "No".
8. Browse for the Ambari SCOM management pack files.

9. Select the following files:

   Ambari.SCOM.Monitoring.mpb
   Ambari.SCOM.Management.mpb
   Ambari.SCOM.Presentation.mpb
   

10. Click "Open"
11. Review the Import list and click "Install".
12. The Ambari SCOM Management Pack installation will start.
    
    

Create Run As Account

Perform the following to configure a account to use when the Ambari SCOM Management Pack talks to the Ambari SCOM Server.

1. After Importing the Management Pack is complete, go to Administration -> Run As Configuration -> Accounts.
2. In the Tasks panel, select "Create Run as Account..."
3. You are presented with the Create Run As Account Wizard.
4. Go thru the wizard, select Run As account type "Basic Authentication".
5. Give the account a Display name and click "Next".
6. Enter the account name and password for the Ambari SCOM Server. This account will be used to connect to the Ambari SCOM Server to access the Ambari REST API. Default is account name is "admin" and password is "admin".
7. Click "Next"
8. Select the "Less secure" distribution security option.
9. Click "Next" and complete the wizard.

Configure the Management Pack

Perform the following to configure the Ambari SCOM Management Pack to talk to the Ambari SCOM Server.

1. Go to Authoring -> Management Pack Templates -> Ambari SCOM
2. In the Tasks panel, select "Add Monitoring Wizard".
3. Select monitoring type "Ambari SCOM"
4. Provide a name and select the destination management pack (see Best Practice).

5. Provide the Ambari URI with is the address of the Ambari SCOM Server in the format:

   http://[ambari-scom-server]:8080/api/
   

   In the above Ambari URI, ambari-scom-server is the Ambari SCOM Server.

6. Select the Run As Account that you created in Create Run As Account.
7. Select "Watcher Node". If node is not listed, click "Add" and browse for the node. Click "Next".
8. Complete the Add Monitoring Wizard and proceed to the Monitoring Scenarios for information on using the management pack.

Best Practice: Create Management Pack for Customizations

By default, Operations Manager saves all customizations such as overrides to the Default Management Pack. As a best practice, you should instead create a separate management pack for each sealed management pack you want to customize.

When you create a management pack for the purpose of storing customized settings for a sealed management pack, it is helpful to base the name of the new management pack on the name of the management pack that it is customizing, such as Ambari SCOM Customizations.

Creating a new management pack for storing customizations of each sealed management pack makes it easier to export the customizations from a test environment to a production environment. It also makes it easier to delete a management pack, because you must delete any dependencies before you can delete a management pack. If customizations for all management packs are saved in the Default Management Pack and you need to delete a single management pack, you must first delete the Default Management Pack, which also deletes customizations to other management packs.