Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

(image copied from https://databricks.com/blog/2015/01/09/spark-sql-data-sources-api-unified-data-access-for-the-spark-platform.html)

 

Spark supports pluggable data sources. We can use make Zeppelin’s  `DistributedResourcePool` a spark data source using Spark DataSource API. Please refer these articles for more information.

 

4.2.1. BaseRelation Implementation

...

ResourcePoolRestAPI class provides APIs to access resources to end-users. Thus it should provide the following functionalities: 

  • list all resources

  • get information for a resource

    • column name, type for tables

    • preview for tables

  • get a resource

    • If the resource is table, it should be downloaded using streaming

...

5. Discussion

5.1. How can a user create TableData instance to share the resource?

...