Status

Current state: "Under DiscussionAccepted"

Discussion thread: http://mail-archives.apache.org/mod_mbox/flink-dev/201712.mbox/%3C8a9d718b-5dae-0fe2-1da6-a8d557d45582%40apache.org%3E

...

We might use usual heap space at the beginning. The internal database can be any JDBC database. External materialization modes (*) are not included in the first version. In the future, Kafka would be read by general Kafka utility functions. Files as well with support for different file systems.

...

 No compatibility changes or other deprecation necessary.

Implementation Plan

1. Basic  Basic Embedded SQL Client Client 

Add the basic features to play around with Flink's streaming SQL.

• Add CLI component that reads the configuration files
  • "Pre-registered table sources"
  • "Job parameters"
• Add executor for retrieving pre-flight information and corresponding CLI SQL parser

  • SHOW TABLES

  • DESCRIBE TABLE

  • EXPLAIN

• Add streaming append query submission to executor

  • Submit jars and run SELECT query using the ClusterClient

  • Collect results on heap and serve them on the CLI side (Internal Mode with SELECT)

  • EXECUTE (for executing a SQL statement stored in a local file)

2. Full Embedded SQL Client

Add important features to fully use the SQL client for a variety of use cases.

• Add support for streaming retract/upsert queries
• Add support for batch queries

• Add user-defined function support

  • Support scalar, aggregate, and table functions
  • Submit corresponding jars to the cluster
  • DESCRIBE FUNCTION
• Add CLI query history
• Add CLI query code completion/highlighting
• Add support for INSERT INTO
  • Read configuration about "Pre-registered table sinks"
  • Allow submission without collect()

3. Discuss/design further features

...