View Source

Project Proposal: Adding trash can for deleted tables

Project Name: Apache Accumulo

Assigned Mentor: Keith Turner

Student: Sreejith Ramakrishnan

Student e-mail: sreejith.code AT gmail DOT com

Project Idea JIRA: https://issues.apache.org/jira/browse/ACCUMULO-1256

1. Abstract

Apache Accumulo is a highly scalable structured store based on Google’s BigTable. Accumulo is written in Java and operates over the HDFS (Hadoop Distributed File System). It is a richer data model than key-value stores. Yet, it is not a fully relational database. Accumulo provides efficient storage and retrieval of structured data and the support for Accumulo tables to be used as input/output in MapReduce operations.

It features automatic load-balancing and partitioning, data compression and fine-grained security labels. The goal of this proposal to enable a trash can facility in Accumulo. When enabled, the tables which are deleted are treated by the rest of Accumulo as deleted. Yet, it should be possible to restore these tables to its original state and name at a later time. They should not appear in client operations that list tables. When an access is attempted to a trash table, a TableDeletedException should be thrown.

2. Detailed description

2.1 New "TRASH" State

For accommodating trashing in Accumulo, a new state called TRASH&nbsp;\[1\] should be added to the existing list of states viz, NEW,  ONLINE,      OFFLINE, DELETING, UNKNOWN. This can be added in&nbsp;

org.apache.accumulo.core.master.state.tables.TableState

2.2 Changes to FATE Operations

Certain behavior of normal FATE operations should be modified to accommodate trash can feature.

2.2.1 Delete Table Operation

When trashing is enabled, the behavior of the delete table FATE operation should be different. After obtaining the table lock, delete table should do the following for a table not in TRASH:

Change the table state to TRASH
Change the table name to something unique that preserves the original table name such as "trash-<origname><yyyymmddhhmmss><count>"
Add table to the timer maintain in master so that after a configurable timeout, the tables in trash are automatically deleted.

Whereas, if the delete table operation is called on a table already in the TRASH state, it should proceed with the normal delete operations.

2.2.2 Undelete Operation

This will require a new fate operation. This fate operation should be similar to the rename table operation, with the additional operation of changing the table state to ONLINE. The original name of the table in the trash can be extracted from its table-name. The table should be renamed to this name.

2.3 Changes in Behaviors

When the master sees a table state of TRASH, the goal state for tablets in that table should be UNASSIGNED. Attempting to initiate an operation on a table in trash should fail. Checks like this are already conducted for offline tables. We can do checks for trash tables in the same place.

The amount of time tables spend in the trash state before really being deleted should be configurable via a system property. Network timing can be unreliable. So, the master can be modified to maintain a timer which keeps track of when each table in the trash should be deleted. Tables can be added to this timer by the delete operation. And on timeout, the timer can simply trigger a delete FATE operation of the corresponding table in the trash.

If a table in trash references certain files, they should not be deleted by the Accumulo garbage collector.

2.4 Configuration Properties

Certain new configuration properties must be supported so that the user can set certain parameters. They could be:

general.trash.enable - Used to enable trashing feature in Accumulo. Can be done at runtime without restarts
general.trash.timeout - Used to set the maximum time after which the tables in trash will be deleted automatically
general.trash.maxspace - Used to set the maximum space allotted for tables in trash. When the space is exceeded, the table with oldest delete timestamp can be discarded to make space.

2.4 API

New methods can be added to the supported table operations. These can be added to the interface TableOperations:

listTrash() - Returns a set of all the tables in trash
undelete(String trashTableName) - Restores the table specified by trashTableName by changing its name back to original and setting its state to ONLINE
undelete(String trashTableName, String tableName) - Restores the table to a new name specified by tableName

3. Timeline

July 22 - August 02

Getting started

Proposal review and discussions, getting familiar with related technologies.

August 05 - September 09

Implementation of new FATE operation undelete and listTrash() and changes to the delete table operation.