Page History

...

Setting up Ambari SCOM assumes the following prerequisite software:

• Ambari SCOM 1.0
  • Apache Hadoop 1.x cluster (HDFS and MapReduce) ¹
• Ambari SCOM 2.

...

• 0
  • Apache Hadoop 2.x cluster (HDFS and YARN/MapReduce) ²
• JDK 1.

...

• 7
• Microsoft SQL Server 2012
• Microsoft JDBC Driver 4.0 for SQL Server

...

• ³
• Microsoft System Center Operations Manager (SCOM) 2012 SP1 or later
• System Center Monitoring Agent installed on Watcher Node

...

• ⁴

¹ Ambari SCOM 1.0 has been tested with a Hadoop cluster based on Hortonworks Data Platform 1.3 for Windows ("HDP 1.3 for Windows")

² Ambari SCOM 2.0 has been tested with a Hadoop cluster based on Hortonworks Data Platform 2.1 for Windows ("HDP 2.1 for Windows")

³ Obtain the Microsoft JDBC Driver 4.0 for SQL Server JAR file (sqljdbc4.jar) at http://technet.microsoft.com/en-us/library/ms378749.aspx ³

⁴ See Microsoft TechNet topic for Managing Discovery and Agents. Minimum Agent requirements .NET 4 and PowerShell 2.0 + 3.0

Package Contents

Anchor
package package

├─ ambari-scom-xversion.y.zzip
├── README.md
├── LICENSE
├── NOTICE
├── server.zip
├── metrics-sink.zip
├── mp.zip
└── ambari-scom.msi └── mp.zip

...

Ambari SCOM Server Installation

Anchor
methods methods

...

1. On each host in the cluster, setup the Hadoop metrics2 interface to use the SQLServerSink.

Edit the hadoop-metrics2.properties file (located in the {C:\hadoop\install\dir}\bin folder of each host in the cluster):

...

*.sink.sql.class=org.apache.hadoop.metrics2.sink.SqlServerSink

namenode.sink.sql.databaseUrl=jdbc:sqlserver://[server]:[port];databaseName=HadoopMetrics;user=[user];password=[password]
datanode.sink.sql.databaseUrl=jdbc:sqlserver://[server]:[port];databaseName=HadoopMetrics;user=[user];password=[password]
jobtracker.sink.sql.databaseUrl=jdbc:sqlserver://[server]:[port];databaseName=HadoopMetrics;user=[user];password=[password]
tasktracker.sink.sql.databaseUrl=jdbc:sqlserver://[server]:[port];databaseName=HadoopMetrics;user=[user];password=[password]
maptask.sink.sql.databaseUrl=jdbc:sqlserver://[server]:[port];databaseName=HadoopMetrics;user=[user];password=[password]
reducetask.sink.sql.databaseUrl=jdbc:sqlserver://[server]:[port];databaseName=HadoopMetrics;user=[user];password=[password]

...

1. Update the Java classpath for each Hadoop service to include the metrics-sink-version.jar and sqljdbc4.jar files.
   
   

   1. Example: Updating the Java classpath for HDP for Windows clusters
      
      The service.xml files will be located in the {C:\hadoop\install\dir}\bin folder of each host in the cluster. The Java classpath is specified for each service in the <arguments> element of the service.xml file.

      Info
      For example, to update the Java classpath for the NameNode component, edit the C:\hadoop\bin\namenode.xml file. Code Block ... <arguments>... -classpath ...;C:\Ambari\metrics-sink-1.5.1.2.5.0.9.0.0-58673.jar;C:\Ambari\sqljdbc4.jar ...</arguments> ...

2. Restart Hadoop for these changes to take affect.

Verify Metrics Collection

1. Confirm metrics are being captured in the SQL Server database by querying the MetricRecord table:

   Code Block
   select * from HadoopMetrics.dbo.MetricRecord

   Info
   In the above SQL statement, HadoopMetrics is the database name.

...

1. Designate a machine in the cluster to run the Ambari SCOM Server.
2. Extract the contents of the server.zip package to obtain the Ambari SCOM Server packages.
   ├── ambari-scom-server-version-conf.zip
   ├── ambari-scom-server-version-lib.zip
   └── ambari-scom-server-version.jar
3. Extract the contents of the ambari-scom-server-version-lib.zip package to obtain the Ambari SCOM dependencies.
4. Extract the contents of the ambari-scom-server-version-conf.zip package to obtain the Ambari SCOM configuration files.

5. From the configuration files, edit the ambari.properties file:

   Code Block
   scom.sink.db.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver scom.sink.db.url=jdbc:sqlserver://[server]:[port];databaseName=HadoopMetrics;user=[user];password=[password]

   Info
   In the above configurationWhere: server = the SQL Server hostname port = the SQL Server port (for example, 1433) user = the SQL Server user (for example, sa) password = the SQL Server password (for example, BigData1)

6. Run the org.apache.ambari.scom.AmbariServer class from the Java command line to start the Ambari SCOM Server.

   Info
   Be sure to include the following in the classpath: ambari-scom-server-version.jar file configuration folder containing the Ambari SCOM configuration files lib folder containing the Ambari SCOM dependencies folder containing the clusterproperties.txt file from the Hadoop install. For example, c:\hadoop\install\dir sqljdbc4.jar SQLServer JDBC Driver file (see Prerequisite Software)

   For example:

   Code Block
   java -server -XX:NewRatio=3 -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit -XX:CMSInitiatingOccupancyFraction=60 -Xms512m -Xmx2048m -cp "c:\ambari-scom\server\conf;c:\ambari-scom\server\lib\*;c:\jdbc\sqljdbc4.jar;c:\hadoop\install\dir;c:\ambari-scom\server\ambari-scom-server-1.5.1.2.5.0.9.0.0-53673.jar" org.apache.ambari.scom.AmbariServer

   Info
   In the above command, be sure to replace the Ambari SCOM version in the ambari-scom-server-version.jar and replace c:\hadoop\install\dir with the folder containing the clusterproperties.txt file.

Verify the Server API

1. From a browser access the API

   Code Block
   http://[ambari-scom-server]:8080/api/v1/clusters

2. Verify that metrics are being reported.

   Code Block
   http://[ambari-scom-server]:8080/api/v1/clusters/ambari/services/HDFS/components/NAMENODE

...

1. Designate a machine in the cluster to run the Ambari SCOM Server.
2. Extract the contents of the server.zip package to obtain the Ambari SCOM Server packages.
3. Run the ambari-scom.msi installer. The "Ambari SCOM Setup" dialog appears:
   

4. Provide the following information:

   Field	Description
   Ambari SCOM package directory	The directory where the installer will place the Ambari SCOM Server packages. For example: C:\Ambari
   SQL Server hostname	The hostname of the SQL Server instance for Ambari SCOM Server to use to store Hadoop metrics.
   SQL Server port	The port of the SQL Server instance.
   SQL Server login	The login username.
   SQL Server password	The login password
   Path to SQL Server JDBC Driver (sqljdbc4.jar)	The path to the JDBC Driver JAR file.
   Path to the cluster layout file (clusterproperties.txt)	The path to the cluster layout properties file.

5. You can optionally select to Start Services
6. Click Install
7. After completion, links are created on the desktop to "Start Ambari SCOM Server", "Browse Ambari API" and "Browse Ambari API Metrics". After starting the Ambari SCOM Server, browse the API and Metrics to confirm the server is working properly.

...

Import the Management Pack

Anchor
mgmtpack-import mgmtpack-import

...

1. Extract the contents of the mp.zip package to obtain the Ambari SCOM management pack (.mpb) files.
2. Ensure Windows Server 2012 running SCOM with SQL Server (full text search).
3. Open System Center Operations Manager.
4. Go to Administration -> Management Packs.
5. From the Tasks panel, select Import Management Packs...
6. In the Import Management Packs dialog, select Add -> Add from disk...
7. You are prompted to search the Online Catalog. Click "No".
8. Browse for the Ambari SCOM management pack files.

9. Select the following files:

   Code Block
   Ambari.SCOM.Monitoring.mpb Ambari.SCOM.Management.mpb Ambari.SCOM.Presentation.mpb

10. Click "Open"
11. Review the Import list and click "Install".
12. The Ambari SCOM Management Pack installation will start.
    
    

Create Run As Account

Anchor
mgmtpack-runas mgmtpack-runas

...

1. Go to Authoring -> Management Pack Templates -> Ambari SCOM
2. In the Tasks panel, select "Add Monitoring Wizard".
3. Select monitoring type "Ambari SCOM"
4. Provide a name and select the destination management pack (see Best Practice).

5. Provide the Ambari URI with is the address of the Ambari SCOM Server in the format:

   Code Block
   http://[ambari-scom-server]:8080/api/

   Info
   In the above Ambari URI, ambari-scom-server is the Ambari SCOM Server.

6. Select the Run As Account that you created in Create Run As Account.
7. Select "Watcher Node". If node is not listed, click "Add" and browse for the node. Click "Next".
8. Complete the Add Monitoring Wizard and proceed to the Monitoring Scenarios for information on using the management pack.

...