...
Code Block |
---|
@Override public void onMessage(Message msg){ // We are interested in only add_partition events on this table. // So, check message type first. if(msg.getStringProperty(HCatConstants.HCAT_EVENT).equals(HCatConstants.HCAT_ADD_PARTITION_EVENT)){ // Retrieve HCatEventMessage, using MessagingUtils. HCatEventMessage hcatMessage = MessagingUtils.getMessage(msg); //Get the actual partition-keys objectfor whichall gotpartitions added. List<Map<String, PartitionString>> partitionpartitionList = (Partition)(((ObjectMessageAddPartitionMessage)msghcatMessage).getObjectgetPartitions()); } } |
You need to have jms jar in your classpath to make this work. You additionally need to have a jms provider’s jar in your classpath as well. Hcatalog uses ActiveMQ as a jms provider. In principle any JMS provider can be used in client side. However, ActiveMQ is recommended. It can be obtained from: http://activemq.apache.org/activemq-550-release.html
Event Message Formats
While HCatalog Event-string formats are pluggable, the strings are in JSON by default. Each event conveys only just enough information to identify the Database/Table/Partition that's been added/deleted in HCatalog. An event-consumer may use the identifiers specified in the event to query HCatalog for further information.
HCatalog sends events for 6 metastore operations. Here's a listing of the supported events, and their corresponding event formats:
1. Creation of Database:
Event type-string: "CREATE_DATABASE"
Example JSON Format:
Code Block |
---|
{
"timestamp" : 1360272556,
"eventType" : "CREATE_DATABASE",
"server" : "hcatserver.mydomain.net",
"servicePrincipal" : "hcat/hcatserver@MYDOMAIN.NET",
"db" : "mydb"
}
|
2. Dropping a Database:
Event type-string: "DROP_DATABASE"
Example JSON Format:
Code Block |
---|
{
"timestamp" : 1360272556,
"eventType" : "DROP_DATABASE",
"server" : "hcatserver.mydomain.net",
"servicePrincipal" : "hcat/hcatserver@MYDOMAIN.NET",
"db" : "mydb"
}
|
3. Creation of a Table:
Event type-string: "CREATE_TABLE"
Example JSON Format:
Code Block |
---|
{
"timestamp" : 1360272556,
"eventType" : "CREATE_TABLE",
"server" : "hcatserver.mydomain.net",
"servicePrincipal" : "hcat/hcatserver@MYDOMAIN.NET",
"db" : "mydb",
"table" : "mytbl"
}
|
4. Dropping a Table:
Event type-string: "DROP_TABLE"
Example JSON Format:
Code Block |
---|
{
"timestamp" : 1360272556,
"eventType" : "DROP_TABLE",
"server" : "hcatserver.mydomain.net",
"servicePrincipal" : "hcat/hcatserver@MYDOMAIN.NET",
"db" : "mydb",
"table" : "mytbl"
}
|
5. Adding (an atomic set of) partitions:
Event type-string: "ADD_PARTITION"
Example JSON Format:
Code Block |
---|
{
"timestamp" : 1360272556,
"eventType" : "ADD_PARTITION",
"server" : "hcatserver.mydomain.net",
"servicePrincipal" : "hcat/hcatserver@MYDOMAIN.NET",
"db" : "mydb",
"table" : "mytbl",
"partitions": [
{ "partKey1" : "partVal1A", "partKey2" : "partVal2A" },
{ "partKey1" : "partVal1B", "partKey2" : "partVal2B" },
{ "partKey1" : "partVal1C", "partKey2" : "partVal2C" }
]
}
|
6. Dropping (a set of) partitions:
Event type-string: "DROP_PARTITION"
Example JSON Format:
Code Block |
---|
{
"timestamp" : 1360272556,
"eventType" : "DROP_PARTITION",
"server" : "hcatserver.mydomain.net",
"servicePrincipal" : "hcat/hcatserver@MYDOMAIN.NET",
"db" : "mydb",
"table" : "mytbl",
"partitions": [
{ "partKey1" : "partVal1A", "partKey2" : "partVal2A" },
{ "partKey1" : "partVal1B", "partKey2" : "partVal2B" },
{ "partKey1" : "partVal1C", "partKey2" : "partVal2C" }
]
}
|
All the JMS messages are sent as TextMessage instances. Apart from the message-body, each message conveys 3 string properties, using the following keys:
- HCatConstants.HCAT_EVENT: The event-type string (E.g. "CREATE_TABLE", "ADD_PARTITIONS", etc.)
- HCatConstants.HCAT_MESSAGE_VERSION: The version-string for the messages (E.g. "0.1", etc.)
- HCatConstants.HCAT_FORMAT: An identifier for the message format (E.g. "json", by default.)