...

The other benefit of this approach is it provide right semantic even if a transaction consists of more than one HMS write request. In that case, there are multiple copies of the cache entry could associate with a single writeid. For example, if there are two alter table request in the transaction, one request route to HMS 1 and the other route to HMS 2. We might not be able to tell which copy is applicable for this writeid if not carefully designed. By using notification log and only make cache available upon commit message, we can make sure we apply all request of the transaction. Once the commit message is processed, we can make sure the copy we have in cache is correct after this transaction.

The other thing to note is this approach does not provide read your own writes. Reads within the transaction cannot request newest copy which contains its own write, since the transaction is not committed thus the cached copy may not be correct. It will be correct until HMS apply all writes of the transaction from notification log. So the reads might end up reading the old copy from cache. I am not sure if there’s a use case read the entry just written in Hive. If it does, we need to cache the written copy in client, so the next time client reads, it will be retrieved from client cache to make sure it contains its own writes.

API changes

When reading, we need to pass a writeid list so HMS can compare it with the cached version. For every commit, we need to tell HMS what is the writeid for the tables changed during the transaction, so HMS can correctly tag the entry with the writeid. We need to add ValidWriteIdList into metastore thrift API and RawStore interfaces, including all table/partition related read calls and commit transaction calls.

...