Versions Compared

Old Version 1

changes.mady.by.user Zhenqiu Huang

Saved on

compared with

New Version 2

changes.mady.by.user Zhenqiu Huang

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Current state: "Under Discussion"

Discussion thread: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSSDiscussion-FlinkFLIP-Python79-UDFFlink-EnvironmentFunction-andDDL-DependencySupport-Management-td33514td33965.html

JIRA: https://issues.apache.org/jira/browse/FLINK-140197151

Released: 

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

...

Before deep into the DDL SQL, we want to discuss the major requirements for defining a function within Flink runtime by related FLIPs:

  1. External lib registration. The requirements come from the hive integration that enhances the adoption of Flink batch. HQL supports syntax like:

    CREATE FUNCTION addfunc AS 'com.example.hiveserver2.udf.add' USING JAR 'hdfs:///path/to/jar'


  2. Language distinguish. Due to bytecode language specifics in Scala, there are some limitations to extract type information from scala function. At the same time, support python UDF in table runtime is another ongoing effort. Thus, the SQL syntax consider support multiple languages. Mysql create function syntax support language in this way:

    CREATE FUNCTION hello (s CHAR(20))RETURNS CHAR(50)DETERMINISTIC RETURN CONCAT('Hello, ',s,'!') LANGUAGE SQL 


  3. Properties Definition. Similar with properties for table, we can add properties for function. The motivation of it is to make the UDF definition more extendable for future works.
  4. Temporary Object Scope.Temporary functions identifiers resolution consider object scopes whether in particular catalog, database or just concurrent catalog and database. Thus, all of the function DDL needs to support 3-part path.

Function DDL SQL Syntax

Create Function Statement

CREATE FUNCTION [IF NOT EXISTS] [catalog_name.db_name.]function_name AS class_name LANGUAGE lang_str [USING resource_path] [WITH PROPERTIES ‘(‘ name=value [, name=value]* ‘)’] 


Drop Function Statement

DROP FUNCTION [IF EXISTS] [catalog_name.][db_name.] function_name;


Alter Function Statement

ALTER FUNCTION [IF EXISTS] [catalog_name.][db_name.] function_name RENAME TO new_name;


Show Function Statement


SHOW FUNCTION  [catalog_name.][db_name]