State
[progress record] :
Proposed time : 2022/03/01
...
4. Link tests of data sources are completed through linkis metastore server service, which now provides corresponding metadata query service
Changes
| Modification | Detail |
---|
1 | Modification of maven module
| - New module linkis-datasource-query-common is added, and new datasource data structure, exception class, and tool class are added
- A new module, linkis-datasource-quwey-server, is added to manage data sources. It provides functions such as adding, deleting, checking, modifying, and testing data sources through restful interfaces
- Added the linkis-metadata-manager-common module, and added the metadata data structure, exception class, and tool class
- The linkis-metadata-manager-server module is added to provide metadata management services and query metadata databases, tables, and columns through restful interfaces
- Added a new linkis-metadata-manager-service-es module to provide the elasticsearch metadata management service
- The linkis-metadata-manager-service-hive module is added to provide the metadata query service for hive
- Add a new module linkis-metadata-manager-service-kafka to provide metadata query service for kafka
- A new module linkis-metadata-manager-service-mysql is added to provide metadata query services for mysql
- A new datasource management Java client module linkis-datasource-client is added to facilitate datasource management using sdk
|
2 | Modification of HTTP interface | - Added the interface for querying metadata d
- New data source add delete change search function
|
3 | Modification of the client interface | LinkisDataSourceRemoteClient interface - GetAllDataSourceTypesResult getAllDataSourceTypes (GetAllDataSourceTypesAction) query all data types
- QueryDataSourceEnvResult queryDataSourceEnv(QueryDataSourceEnvAction) Queries the cluster configurations that can be used by the data source
- GetInfoByDataSourceIdResult getInfoByDataSourceId (GetInfoByDataSourceIdAction) : through the data source id query data source information
- QueryDataSourceResult QueryDataSourceAction (QueryDataSourceAction) Queries data sources
- GetConnectParamsByDataSourceIdResult getConnectParams (GetConnectParamsByDataSourceIdAction) get connection configuration parameters
- CreateDataSourceResult createDataSource(CreateDataSourceAction) Creates a data source
- DataSourceTestConnectResult getDataSourceTestConnect (DataSourceTestConnectAction) to test whether or not the data source connection is established properly
- DeleteDataSourceResult deleteDataSource(DeleteDataSourceAction) Deletes a data source
- ExpireDataSourceResult expireDataSource(ExpireDataSourceAction) Sets the data source to the expired state
- GetDataSourceVersionsResult getDataSourceVersions (GetDataSourceVersionsAction) query list data source configuration version
- PublishDataSourceVersionResult publishDataSourceVersion (PublishDataSourceVersionAction) released data source configuration version
- UpdateDataSourceResult UpdateDataSourceAction (UpdateDataSourceAction) Updates data sources
- UpdateDataSourceParameterResult updateDataSourceParameter (UpdateDataSourceParameterAction) to update the data source configuration parameters
- GetKeyTypeDatasourceResult getKeyDefinitionsByType (GetKeyTypeDatasourceAction) data source type of a query need configuration properties
LinkisMetaDataRemoteClient interface - MetadataGetDatabasesResult getDatabases (MetadataGetDatabasesAction) query the database list
- MetadataGetTablesResult getTables(MetadataGetTablesAction) Queries table data
- MetadataGetTablePropsResult getTableProps (MetadataGetTablePropsAction)
- MetadataGetPartitionsResult getPartitions (MetadataGetPartitionsAction) query partition table
- MetadataGetColumnsResult getColumns(MetadataGetColumnsAction) Queries the columns of the data table
|
4 | Modification of database table structure | - No table to modify
- The new table structure is as follows:
Image Modified |
5 | Modification of configuration item |
|
6 | Modification Error code |
|
7 | Modifications for Third Party Dependencies |
|
Compatibility, Deprecation, and Migration Plan
...