Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Page properties


Document the state by adding a label to the FLIP page with one of "discussion", "accepted", "released", "rejected".

Discussion threadhere (<- link to https://lists.apache.org/list.html?dev@flink.apache.org)thread/bto7mpjvcx7d7k86owb00dwrm65jx8cn
Vote threadhere (<- link to https://lists.apache.org/list.html?dev@flink.apache.org)
JIRAhere (<- link to https://issues.apache.org/jira/browse/FLINK-XXXX)
Release<Flink Version>

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

Describe the problems you are trying to solve.

Public Interfaces

Proposed Changes

Describe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgement based on the scope of the change.

Compatibility, Deprecation, and Migration Plan

  • What impact (if any) will there be on existing users?
  • If we are changing behavior how will we phase out the older behavior?
  • If we need special migration tools, describe them here.
  • When will we remove the existing behavior?

Test Plan

Describe in few sentences how the FLIP will be tested. We are mostly interested in system tests (since unit-tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

Rejected Alternatives

thread/m3hw5cc1ovtvvplv9hpd56fzzl44xxcr
JIRA

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyFLINK-34054

Release1.19


Table of Contents

Motivation

Currently, when users call a function or call a procedure, they must specify all fields in order. When there are a large number of parameters, it is easy to make mistakes and cannot omit specifying non-mandatory fields.

By using named parameters, you can selectively specify the required parameters, reducing the probability of errors and making it more convenient to use.

Public Interfaces

Interface change

Introduce a new annotation to specify the parameter name, indicate if it is optional, and potentially support specifying default values in the future.

Deprecate the argumentNames method in FunctionHints as it is not user-friendly for specifying argument names with optional configuration.

Code Block
languagejava
public @interface ArgumentHint {
    /**
     * The name of the parameter, default is an empty string.
     */
    String name() default "";

    /**
     * Whether the parameter is optional, default is false.
     */
    boolean isOptional() default false;

    /**
     * The data type hint for the parameter.
     */
    DataTypeHint type() default @DataTypeHint();
}


Code Block
public @interface FunctionHint {
 
    /**
     * Deprecated attribute for specifying the names of the arguments.
     * It is no longer recommended to use this attribute.
     */
    @Deprecated
    String[] argumentNames() default {""};
 
    /**
     * Attribute for specifying the hints and additional information for function arguments.
     */
    ArgumentHint[] arguments() default {};
}


Code Block
public @interface ProcedureHint {
 
    /**
     * Deprecated attribute for specifying the names of the arguments.
     * It is no longer recommended to use this attribute.
     */
    @Deprecated
    String[] argumentNames() default {""};
 
    /**
     * Attribute for specifying the hints and additional information for procedure arguments.
     */
    ArgumentHint[] arguments() default {};
}

Develop functions or call procedures that support named parameters

The UDX or procedure class  that support named parameters  can only have one method, and users can optionally specify parameters when calling functions or procedures.

And we can use ArgumentHint to specific the argument name and indicate if argument is optional.

Code Block
languagejava
// UDF Development

public static class NamedArgumentsTableFunction extends TableFunction<Object> {

   // Example usage: SELECT * FROM TABLE(my_table_function(in1 => 'value1', in2 => 'value2'))

   // Example usage: SELECT * FROM TABLE(my_table_function(in1 => 'value1', in2 => 'value2', in3 => 'value3'))

   @FunctionHint(
           output = @DataTypeHint("STRING"),             
           arguments = {
                @ArgumentHint(name = "in1", isOptional = true, type = @DataTypeHint("STRING")),
                @ArgumentHint(name = "in2", isOptional = true, type = @DataTypeHint("STRING"))
                @ArgumentHint(name = "in3", isOptional = true, type = @DataTypeHint("STRING"))})      
    public void eval(String arg1, String arg2, String arg3) {
       collect(arg1 + ", " + arg2 + "," + arg3);
   }

}


// Call Procedure Development

public static class NamedArgumentsProcedure implements Procedure {

   // Example usage: CALL myNamedProcedure(in1 => 'value1', in2 => 'value2')

   // Example usage: CALL myNamedProcedure(in1 => 'value1', in2 => 'value2', in3 => 'value3')

   @ProcedureHint(
           output = @DataTypeHint("STRING"),             
		   arguments = {
                @ArgumentHint(name = "in1", isOptional = false, type = @DataTypeHint("STRING")),
                @ArgumentHint(name = "in2", isOptional = true, type = @DataTypeHint("STRING"))
                @ArgumentHint(name = "in3", isOptional = true, type = @DataTypeHint("STRING"))})       
   public String[] call(ProcedureContext procedureContext, String arg1, String arg2, String arg3) {
       return new String[]{arg1 + ", " + arg2 + "," + arg3};
   }
}


Call functions or procedures using named parameters

Users can call functions or procedures with named parameters as follows:

Code Block
languagesql
-- for scalar function
SELECT my_scalar_function(param1 => ‘value1’, param2 => ‘value2’’) FROM []


-- for table function
SELECT  *  FROM TABLE(my_table_function(param1 => 'value1', param2 => 'value2'))

-- for agg function
SELECT my_agg_function(param1 => 'value1', param2 => 'value2') FROM []



-- for call procedure
CALL  procedure_name(param1 => ‘value1’, param2 => ‘value2’)


Proposed Changes

Implementing Named Parameters:

In FLIP-65, UDX has already supported the ability to parse argument names from FunctionHint and ProcedureHint.  Calcite has long supported the ability to use named optional and default arguments when calling functions (CALCITE-941), but some issues and limitations were discovered during the poc.

1. Passing Named Parameters to the Calcite:

Due to Calcite's behavior of reordering operands based on named arguments, the reordering relies on the implementation of the SqlOperandTypeCheck interface in order to implement the SqlOperandMetadata interface. Therefore, it is necessary to extend the TypeInferenceOperandChecker class for this specific implementation. 

2.Fixing Calcite Bugs Related to Named Parameters:

During POC verification, bugs were discovered in Calcite that caused issues during the validation phase. We need to modify the SqlValidatorImpl and SqlToRelConverter to address these problems.

Capabilities and Limitations

Capabilities

  • Reflection-based Named Parameters:

When using the feature without explicitly specifying parameter hints, the function will automatically retrieve the parameter names using reflection, allowing them to be used as named parameters.

  • Optional Argument Names:

If the argument names specified in the hint do not match the actual parameter names, it will fall back to using default values.   When users use optional argument names, we need corresponding UDX methods without overloads to avoid function conflicts caused by assigning default values.

Limitations:

  • Variable arguments are not supported with named parameters.
  • the UDX or procedure class  that support named parameters  can only have one eval method
  • Due to the current limitations of Calcite-947[1], we are unable to specify a default value for omitted parameter. The default value for omitted parameters is Null.

Compatibility, Deprecation, and Migration Plan

Compatibility:

Since named parameters are a newly introduced feature, there are no compatibility issues. However, it's important to note that functions registered using the deprecated TableEnvironment.registerFunction API will not support named parameters.

Deprecation:

No deprecation is required for this feature.

Migration Plan:

No migration is needed as this is a new feature.

Test Plan

To validate this feature, we can develop custom UDFs or call procedures and use named parameters when invoking them.

Rejected Alternatives

None


1 . https://issues.apache.org/jira/browse/CALCITE-947

2. poc: https://github.com/apache/flink/compare/master...hackergin:flink:poc_named_argumentIf there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other way.