Status

Current state:  [One of "Under Discussion", " Accepted", "Rejected"]

Discussion thread:  here (<- link to https://mail-archiveslists.apache.org/mod_mbox/flink-dev/)
JIRA: here (<- link to thread/m4r3wrd7p96wdst3nz3ncqzog6kf51cf

Vote thread: https://issueslists.apache.org/thread/jira/browse/FLINK-XXXX)fosvz0zcyfn6bp6vz2oxl45vq9qhkn2v

JIRA:

Released: <Flink Version>1.18.0

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

The TRUNCATE TABLE statement is a SQL command that allows users to quickly and efficiently delete all rows from a table without dropping the table itself.  This statement is commonly used in data warehouse, where large data sets are frequently loaded and unloaded from tables. It can improve performance by reducing the amount of time and resources required to delete large amounts of data.

Also, the TRUNCATE TABLE statement can help simlify and streamling data management tasks by eliminating the need for complex queries or data manipulation operations.

  Considering the TRUNCATE TABLE statment is useful and widely-used, this FLIP is aimed to support TRUNCATE TABLE  statment in Flink. This FLIP will bring Flink the TRUNCATE TABLE  syntax and an interface with which the coresponding connectors can implement their own  logic for truncating table.

Note:  Considering the use cases of truncate table are mainly for batch scenario and the semantic in stream scenario should be discussed separately, this FLIP to to support truncate table in batch only.

Public Interfaces

...

Briefly list any new interfaces that will be introduced as part of this proposal or any existing interfaces that will be removed or changed. The purpose of this section is to concisely call out the public contract that will come along with this feature.

A public interface is any change to the following:

• DataStream and DataSet API, including classes related to that, such as StreamExecutionEnvironment

Proposed Changes

Syntax

We propose add the following syntax for TRUNCATE TABLE statement.

Code Block
language sql
TRUNCATE TABLE table_name

It supports trucate permanent or temporary table,  if it's a view table, it should throw TableException with message Truncate a view table is not support .

Public interfaces Changes

We propose add a inerface for TRUNCATE TABLE statment. The connectors can implement this interface to custom their logic for truncating table.

Code Block
language java
/** * Enables to delete all existing data in a {@link DynamicTableSink} table using {@code TRUNCATE * TABLE} statement. * * <p>For {@code TRUNCATE TABLE} statement, if the corresponding {@link DynamicTableSink} have * implemented this interface, then the method {@link #truncateTable()} will be invoked in execution * phase. Otherwise, Flink will throw an exception directly. */ @PublicEvolving public interface SupportsTruncate { /** * Execute truncating table. * * Note: please remember to throw exception if the truncation hasn't been executed successfully, * otherwise it'll be still considered to haven been executed successfully by Flink. */ void executeTruncation(); }

Proposed Changes

1: Add a parse rule in parserImpls.ftl to match TRUNCATE TABLE  statement and convert it to SqlTruncateTable 

2: Create a SqlTruncateTableConveter  to TruncateTableOperation, and register SqlTruncateTableConveter to SqlNodeConverters.

3: TruncateTableOperation  should implement ExecutableOperation, and in method #execute(Context ctx), it will  get the DynamicTableSink  of the table which should implement SupportsTruncate   and then call method SupportsTruncate#executeTruncation  directly to truncate the table. If the DynamicTableSink  doesn't implemente this interface, it should throw TableException('The truncate statement for the table is not supported as it hasn't implemented the interface SupportsTruncate') . For some sinks that doesn't support deleting data, it can also implements this interface but throw more concrete exception like "xxx donesn't support to truncate a table as delete is impossible for xxx"Describe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgement based on the scope of the change.

Compatibility, Deprecation, and Migration Plan

• What impact (if any) will there be on existing users? 
• If we are changing behavior how will we phase out the older behavior? 
• If we need special migration tools, describe them here.
• When will we remove the existing behavior?

Test Plan

No any compatibility issue.

Test Plan

It'll be covered by UT & IT.Describe in few sentences how the FLIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

Rejected Alternatives

If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.