Motivation

Current SQL engine has a number of critical limitations:

1. No data co-location control, i.e. arbitrary data can be returned silently
2. Limited execution modes: either two-phase execution (default) or "distributed joins" which adds one more phase
3. Lack of proper planner which will take in count both data distribution and data statistics

This document is concerned only with engine's infrastructure. Query planning and statistics are non goals.

Description

What to take in count:

• Partition reservation - all necessary partition should be reserved and released properly for all stages
• Cancellation - it should be possible to cancel any query type
• Streaming - it should be possible to send many packets to destination nodes in advance because 
• Reactivity - all query stages should operate in non-blocking fashing, no waits should occur inside query pool threads
• Manageability - it should be possible to get clear picture of query plan and influence some stages with query hints
• Pipeline-agnostic - design should allow us to have both classical ("volcano") and code-generated pipelines in future
• Result-agnostic - it should be possible to work with both heap, offheap and out-process intermediate results

"Source" - fundamental concept. Types:

• Local - getting local node's data
• Ephemeral - any kind of temporal data retrieved from anywhere. This might be: intermediate query results, results of non-correlated subquery, results of REPLICATED LEFT JOIN PARTITIONED)

Risks and Assumptions

// TODO: Describe project risks, such as API or binary compatibility issues, major protocol changes, etc.

Discussion Links

// TODO: Links to discussions on the devlist, if applicable.

Reference Links

// TODO: Links to various reference documents, if applicable.

Tickets

// TODO: Links or report with relevant JIRA tickets.