Versions Compared

Old Version 4

changes.mady.by.user Arup Malakar

Saved on

compared with

New Version 5

changes.mady.by.user Arup Malakar

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The client in order to receive message must be subscribed to the right topic. Topic name for different types of events are different and is configurable in few cases. All default topic names are prefixed with a string which is configurable in hive-site.xml. The configuration key is hcat.msgbus.topic.prefix which defaults to hcat. We shall refer to the value of the topic prefix as HCAT_TOPIC_PREFIX in rest of the document. For example drop_database events would be sent to topic HCAT_PREFIX (e.g. hcat). add_partition messages would be sent to <HCAT_PREFIX>.<DB_NAME>.<TABLE_NAME> (eg: hcat.default.example_table). But the topic name for a table could be changed per table by user. For example following hive query sets the topic name for a table example_table to the string example_table_topic_name:

Code Block

alter table set properties ("hcat.msgbus.topic.name"=example_table_topic_name)

...
How to receive notification

...

Event type-string: "CREATE_DATABASE"
Topic Name: HCAT_PREFIX
Example JSON Format:

Code Block
{
  "timestamp" : 1360272556,
  "eventType" : "CREATE_DATABASE",
  "server"    : "hcatserver.mydomain.net",
  "servicePrincipal" : "hcat/hcatserver@MYDOMAIN.NET",
  "db"        : "mydb"
}

...

Event type-string: "DROP_DATABASE"
Topic Name: HCAT_PREFIX
Example JSON Format:

Code Block
{
  "timestamp" : 1360272556,
  "eventType" : "DROP_DATABASE",
  "server"    : "hcatserver.mydomain.net",
  "servicePrincipal" : "hcat/hcatserver@MYDOMAIN.NET",
  "db"        : "mydb"
}

...

Event type-string: "CREATE_TABLE"
Topic Name: HCAT_PREFIX.DB_NAME
Example JSON Format:

Code Block
{
  "timestamp" : 1360272556,
  "eventType" : "CREATE_TABLE",
  "server"    : "hcatserver.mydomain.net",
  "servicePrincipal" : "hcat/hcatserver@MYDOMAIN.NET",
  "db"        : "mydb",
  "table"     : "mytbl" 
}

...

Event type-string: "DROP_TABLE"
Topic Name: HCAT_PREFIX.DB_NAME
Example JSON Format:

Code Block
{
  "timestamp" : 1360272556,
  "eventType" : "DROP_TABLE",
  "server"    : "hcatserver.mydomain.net",
  "servicePrincipal" : "hcat/hcatserver@MYDOMAIN.NET",
  "db"        : "mydb",
  "table"     : "mytbl" 
}

...

Event type-string: "ADD_PARTITION"
Topic Name: HCAT_PREFIX.DB_NAME.TABLE_NAME (default) but is user configurable
Example JSON Format:

Code Block
{
  "timestamp" : 1360272556,
  "eventType" : "ADD_PARTITION",
  "server"    : "hcatserver.mydomain.net",
  "servicePrincipal" : "hcat/hcatserver@MYDOMAIN.NET",
  "db"        : "mydb",
  "table"     : "mytbl",
  "partitions": [
                   { "partKey1" : "partVal1A", "partKey2" : "partVal2A" },
                   { "partKey1" : "partVal1B", "partKey2" : "partVal2B" },
                   { "partKey1" : "partVal1C", "partKey2" : "partVal2C" }
                ]
}

...

Event type-string: "DROP_PARTITION"
Topic Name: HCAT_PREFIX.DB_NAME.TABLE_NAME (default) but is user configurable
Example JSON Format:

Code Block
{
  "timestamp" : 1360272556,
  "eventType" : "DROP_PARTITION",
  "server"    : "hcatserver.mydomain.net",
  "servicePrincipal" : "hcat/hcatserver@MYDOMAIN.NET",
  "db"        : "mydb",
  "table"     : "mytbl",
  "partitions": [
                   { "partKey1" : "partVal1A", "partKey2" : "partVal2A" },
                   { "partKey1" : "partVal1B", "partKey2" : "partVal2B" },
                   { "partKey1" : "partVal1C", "partKey2" : "partVal2C" }
                ]
}

...