DistributedResourcePool (expanded view)

SparkInterpreter

RemoteInterpreterServer

DistributedResourcePool

RemoteInterpreterEventClient
(ResourcePoolConnector)

LocalResource1

RemoteResource1
(for JDBC)

RemoteResource2
(for ...)

Spark Cluster

Spark Cluster

Spark Cluster

ZeppelinResourcePoolDataSource

use zeppelin's distributed resource pool as spark data source

SparkInterpreter can put `SparkTableData` into the resource pool

LocalResource 1

LocalResource 1

LocalResource 1

Resource N

LocalResource 1

LocalResource 1

LocalResource 1

InterpreterResultTableData

local resources
(created in this interpreter)

LocalResource 1

LocalResource 1

LocalResource 1

SparkTableData N

LocalResource 1

LocalResource 1

LocalResource 1

RemoteResource N

remote resources
(created from other interpreters)

Spark Cluster

Table C

Table C

Table C

RDD N

Some interpreters might not be connected with a storage
(e.g shell interpreter)
In this case, they might persist `TableData` in memory directly

LocalResource 1

LocalResource 1

LocalResource 1

SparkSQLTableData N

Depending on implementation,

`TableData` might keep the query only
to reproduce the table result,
but not to affect on the external storage

SparkInterpreter can use `TableData` instances
created from other interpreters using
remote method invocation such as
`rows()`, `columns()`