MapReduce Job — POST mapreduce/jar
Table of Contents |
---|
Description
Creates and queues a standard Hadoop MapReduce job.
Info | ||
---|---|---|
| ||
As of Hive 0.13.0, GET version/hadoop displays the Hadoop version used for the MapReduce job. |
URL
http://
www.myserver.com/templeton/v1/mapreduce/jar
...
Name | Description | Required? | Default |
---|---|---|---|
jar | Name of the jar file for Map Reduce to use. | Required | None |
class | Name of the class for Map Reduce to use. | Required | None |
libjars | Comma separated jar files to include in the classpath. | Optional | None |
files | Comma separated files to be copied to the map reduce cluster. | Optional | None |
arg | Set a program argument. | Optional | None |
define | Set a Hadoop configuration variable using the syntax | Optional | None |
statusdir | A directory where WebHCat will write the status of the Map Reduce job. If provided, it is the caller's responsibility to remove this directory when done. | Optional | None |
enablelog | If statusdir is set and enablelog is "true", collect Hadoop job configuration and logs into a directory named
This parameter was introduced in Hive 0.12.0. (See HIVE-4531.) | Optional in Hive 0.12.0+ | None |
callback | Define a URL to be called upon job completion. You may embed a specific job ID into this URL using | Optional | None |
usehcatalog | Specify that the submitted job uses HCatalog and therefore needs to access the metastore, which requires additional steps for WebHCat to perform in a secure cluster. (See HIVE-5133.) This parameter will be introduced in Hive 0.13.0. | Optional in Hive 0.13.0+ | false |
The standard parameters are also supported.
Results
Name | Description |
---|---|
id | A string containing the job ID similar to "job_201110132141_0001". |
info | A JSON object containing the information returned when the job was queued. See the Hadoop documentation ( |
Example
Code and Data Setup
No Format |
---|
% hadoop fs -put wordcount.jar .
% hadoop fs -put transform.jar .
% hadoop fs -ls .
Found 2 items
-rw-r--r-- 1 ctdean supergroup 23 2011-11-11 13:29 /user/ctdean/wordcount.jar
-rw-r--r-- 1 ctdean supergroup 28 2011-11-11 13:29 /user/ctdean/transform.jar
|
Curl Command
No Format |
---|
% curl -s -d user.name=ctdean \ -d jar=wordcount.jar \ -d class=org.myorg.WordCount \ -d libjars=transform.jar \ -d arg=wordcount/input \ -d arg=wordcount/output \ 'http://localhost:50111/templeton/v1/mapreduce/jar?user.name=ekoifman' |
Info | ||
---|---|---|
| ||
Prior to Hive 0.13.0, user.name was specified in POST requests as a form parameter: In Hive 0.13.0 onward, user.name should be specified in the query string (as shown above): |
JSON Output
No Format |
---|
{
"id": "job_201111121211_0001",
"info": {
"stdout": "templeton-job-id:job_201111121211_0001
",
"stderr": "",
"exitcode": 0
}
}
|
...
Panel | ||||||
---|---|---|---|---|---|---|
| ||||||
Previous: POST mapreduce/streaming General: WebHCat Reference – WebHCat (Templeton) Manual – HCatalog Manual – Hive Home |