Status
Current state: [One of "Under Discussion", " Accepted", "Rejected"]
Discussion thread: here (<- link to https://mail-archiveslists.apache.org/mod_mbox/flink-dev/)
JIRA: here (<- link to thread/m4r3wrd7p96wdst3nz3ncqzog6kf51cf
Vote thread: https://issueslists.apache.org/thread/jira/browse/FLINK-XXXX)fosvz0zcyfn6bp6vz2oxl45vq9qhkn2v
JIRA:
Jira | ||||||
---|---|---|---|---|---|---|
|
Released: <Flink Version>1.18.0
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
Motivation
The TRUNCATE TABLE statement is a SQL command that allows users to quickly and efficiently delete all rows from a table without dropping the table itself. This statement is commonly used in data warehouse, where large data sets are frequently loaded and unloaded from tables. It can improve performance by reducing the amount of time and resources required to delete large amounts of data.
...
Considering the TRUNCATE TABLE statment is useful and widely-used, this FLIP is aimed to support TRUNCATE TABLE statment in Flink. This FLIP will bring Flink the TRUNCATE TABLE
syntax and an interface with which the coresponding connectors can implement their own logic for truncating table.
Note: Considering the use cases of truncate table are mainly for batch scenario and the semantic in stream scenario should be discussed separately, this FLIP to to support truncate table in batch only.
Public Interfaces
...
Briefly list any new interfaces that will be introduced as part of this proposal or any existing interfaces that will be removed or changed. The purpose of this section is to concisely call out the public contract that will come along with this feature.
A public interface is any change to the following:
- DataStream and DataSet API, including classes related to that, such as StreamExecutionEnvironment
- Classes marked with the @Public annotation
- On-disk binary formats, such as checkpoints/savepoints
- User-facing scripts/command-line tools, i.e. bin/flink, Yarn scripts, Mesos scripts
- Configuration settings
- Exposed monitoring information
Proposed Changes
Syntax
We propose add the following syntax for TRUNCATE TABLE statement.
Code Block | ||
---|---|---|
| ||
TRUNCATE TABLE table_name |
It supports trucate permanent or temporary table, if it's a view table, it should throw TableException
with message Truncate a view table is not support
.
Public interfaces Changes
We propose add a inerface for TRUNCATE TABLE statment. The connectors can implement this interface to custom their logic for truncating table.
Code Block | ||
---|---|---|
| ||
/**
* Enables to delete all existing data in a {@link DynamicTableSink} table using {@code TRUNCATE
* TABLE} statement.
*
* <p>For {@code TRUNCATE TABLE} statement, if the corresponding {@link DynamicTableSink} have
* implemented this interface, then the method {@link #truncateTable()} will be invoked in execution
* phase. Otherwise, Flink will throw an exception directly.
*/
@PublicEvolving
public interface SupportsTruncate {
/**
* Execute truncating table.
*
* Note: please remember to throw exception if the truncation hasn't been executed successfully,
* otherwise it'll be still considered to haven been executed successfully by Flink.
*/
void executeTruncation();
} |
Proposed Changes
1: Add a parse rule in parserImpls.ftl to match TRUNCATE TABLE
statement and convert it to SqlTruncateTable
2: Create a SqlTruncateTableConveter
to TruncateTableOperation, and register
SqlTruncateTableConveter to
SqlNodeConverters.
3: TruncateTableOperation
should implement ExecutableOperation, and in method
get the #execute(Context ctx), it will
DynamicTableSink
of the table which should implement SupportsTruncate
and then call method SupportsTruncate#executeTruncation
directly to truncate the table. If the DynamicTableSink
doesn't implemente this interface, it should throw TableException('The truncate statement for the table is not supported as it hasn't implemented the interface SupportsTruncate')
. For some sinks that doesn't support deleting data, it can also implements this interface but throw more concrete exception like "xxx donesn't support to truncate a table as delete is impossible for xxx"Describe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgement based on the scope of the change.
Compatibility, Deprecation, and Migration Plan
- What impact (if any) will there be on existing users?
- If we are changing behavior how will we phase out the older behavior?
- If we need special migration tools, describe them here.
- When will we remove the existing behavior?
Test Plan
No any compatibility issue.
Test Plan
It'll be covered by UT & IT.Describe in few sentences how the FLIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?
Rejected Alternatives
If there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.