Page History

...

For a non-partitioned table, the lock modes are pretty intuitive. When the table is being read, a S
lock is acquired, whereas an X lock is acquired for all other operations (insert into the table,
alter table of any kind etc.)

...

A 'S' lock on table and relevant partition is acquired when a read is being performed.
For all other operations, an 'X' lock is taken on the partition. However, if the change is
only applicable to the newer partitions, a 'S' lock is acquired on the table, whereas if the
change is applicable to all partitions, a 'X' lock is acquired on the table. Thus, older
partitions can be read and written into, while the newer partitions are being converted to
RCFile. Whenever a partition is being locked in any mode, all its parents are locked in 'S' mode.

...

In order to avoid deadlocks, a very simple scheme is proposed here. All the objects to be locked
are sorted lexicographically, and the required mode lock is acquired. Note that in some cases, the
list of objects may not be known - for eg. in case of dynamic partitions, the list of partitions
being modified is not known at compile time - so, the list is generated conservatively. Since the
number of partitions may not be known, an exclusive lock is taken on the table, or the prefix that
is known.

Two new configurable parameters will be added to decide the number of retries for the lock and the
wait time between each retry. If the number of retries are really high, it can lead to a live lock.
Look at ZooKeeper recipes (http://hadoop.apache.org/zookeeper/docs/r3.1.2/recipes.html#sc_recipes_Locks) to
see how read/write locks can be implemented using the zookeeper apis. Note that instead of waiting,
the lock request will be denied. The existing locks will be released, and all of them will be retried
after the retry interval.

...