...

dfdl:choiceDispatchKey="{ sdf:replace(xs:string(../MessageTextFormatIdentifier), ' ', '_') }"

The function class would look something like the below

@FunctionClassInfo(
		name =
 "replace", 
		namespace = "com.ext.UDFunction.StringFunctions" 
)
public class Replace {
	public String evaluate(String orig, String pre, String post) {
		//implementation...
	}
}

Another use case would be implementing normalizing the normalization of elevation above Mean-Sea-Level (MSL) to Height-Above-Ellipsoid (HAE) for Link16F1 data. In the DFDL schema, we might call something like the below; where the functions will return the result of the conversion.

dfdl:outputValueCalc="{ mhdf:convert_to_hae(
	xs:int(../elevation), 
	xs:int(../hae_adjustment, ../scaling_factor) }"

The function class would look something like the below

@FunctionClassInfo(
		name = "convert_to_hae", 
	xs:float(../scaling_factor)) }"	namespace = "http://extOther.UDFunction.ElevationConversions.com" 
)
public class MSLConversions {
	public Float evaluate(Integer elevation_msl_raw, Integer hae_adjustment_raw, Float scaling_factor) {
		//implementation..
	}
}

Requirements

1. The UDF will be defined in a stand-alone library outside of the Daffodil codebase
2. The UDF must be accessible to and callable from the Daffodil code
3. Daffodil must be able to process and pass the return value from the UDF back to the Schema
4. The support of UDFs in the DFDL Schema must be language agnostic and not Java, Scala or Daffodil specific

...

The Daffodil solution will use a combination of JAVA's ServiceLoader and Reflection APIs.

Daffodil Provided Jar

Daffodil will provide a UDFunctionProvider abstract class and an annotation class FunctionClassInfo.

The UDFunctionProvider class will have a private class array for all the function classes the provider is aware of, and its setter and getter functions, as well as a lookupFunctionClass method that will return a function class object. The implementer is expected to extend this class for each provider class they supply.

The FunctionClassInfo annotation class must be applied and filled in for each function class. It provides name and namespace elements that must be filled out for the function class to be properly identified by the provider when queried by Daffodil .

UDF Implementation

The implementer will be expected to implement at least 2 classes: a provider class and at least one function class.

The provider class will be an implementation of the Daffodil provided UDFuntionProvider class. It will contain . One will be a "provider" class containing a lookup function , that returns an initialized object based on a supplied name and namespace. This "provider" class will act as a traditional service provider as explained in the ServiceLoader API, and should have a src/META-INF/services folder in its JAVA project that contains the fully qualified name of the UDFunctionProvider class that we supply. Through thisthat file, this class will be made visible to the ServiceLoader API and the function class object can be returned to Daffodil.   This class will also be expected to have access to all the function classes it's responsible for, maintain a list of these classes  and be able to initialize them as needed. This class can also provide state to each function class it is in charge of, for example, a database connection or perhaps a counter. A sample is provided below.

public class myUDFunctionProvider extends UDFunctionProvider {
	public Objext myUDFunctionProvider() {
		super.setFunctionClasses(new Class<?>[] { ns1_func1.class, ns2_func2.class, ns1_func3.class });
	}

	public Object lookupFunctionClass(String namespace, String name) {
		switch( String.join("_", namespace, name)) {
			case "ns1_func1":
				return new func1();
			case "ns2_func2":
				return new func2();
			case "ns1_func3":
				return new func3_randomName();
			default:
				return null;
		}
	}
}

The rest of the classes function classes  will contain the functionality of the UDF embodied in an evaluate method. This class is what will be initialized and returned by the provider class. This UDF The function class will be expected to implement 3 methods as seen belowan evaluate method as well as apply the Daffodil provided functionClassInfo annotation to the class . Because the parameter types and the return types of the evaluate function are dependent on the functionality, and we really only care about the name, we will not provide an abstract class for it. Each function that the implementer wishes to expose must be defined in a class with an evaluate function, and must be available to the lookup function.

...

...

See Proposal: User Defined Functions for sample function class.

Daffodil Service Loader

Daffodil will use the ServiceLoader API to poll for UDF Provider classes and return the desired function class on request.

As mentioned above, we will provide an abstract class for UDF "Provider" class. It will contain one abstract lookup function class method . This class will be externally available for the implementer to use. Daffodil will  also have an internal class that will implement a function that calls the ServiceLoader iterator, and call the lookupFunctionClass method previously mentioned. If not Daffodil will have an internal Java class that implements a function that uses the ServiceLoader iterator to aggregate all the provider classes, the function classes list they provide, and the lookupFunctionClass method they implement. This class will have its own function class list, which is aggregated from all function classes of all providers it is aware of. It will also have a map which links names and namespaces from each function class to its provider object. Its lookupFunctionClass method will use this map to lookup the right provider class to call to get the function class object. If no function class found, we will return null, else we will return the desired object. This internal class will be implemented in Java as the ServiceLoader proved difficult to use in Scala.

public Object lookupFunctionClass(String namespace, String name) {
	Object funcObj = null;

	try {
		for (UDFunctionProvider fp : loader) {
			funcObj = fp.lookupFunction(this.functionProviderLookup.get(String.join("_", namespace, name));
			if (funcObjfp != null ) {
				breakfuncObj = fp.lookupFunctionClass(namespace, name);
			}
		}
	} catch (Exception e) {
		e.printStackTrace();
	}
	return funcObj;
}

...

case class UDFunctionCallExpr(nameAsParsed: String, fnQName: RefQName, args: List[Expression],
resultType: NodeInfo.Kind, argsTypes: List[NodeInfo.Kind], constructor: List[CompiledDPath] => RecipeOp)
	extends FunctionCallBase(nameAsParsed, fnQName, args) {

	lazy val argsToArgType = (args zip argsTypes).toMap
	override lazy val inherentType = resultType

	override def targetTypeForSubexpression(childExpr: Expression): NodeInfo.Kind = {
		argsToArgType.get(childExpr) match {
			case Some(argType) => argType
			case None => {
				Assert.invariantFailed("subexpression %s is not an argument.".format(subexp))
			}
		}
	}
	override lazy val compiledDPath = {
		val recipes = args.map { _.compiledDPath }
		val res = new CompiledDPath(constructor(recipes) +: conversions)
		res
	}
}

case class UDFunctionCall(recipes: List[CompiledDPath], functionClassObject: Object, udfEvaluate: Method)
	extends FNArgsList(recipes) {

	override def computeValues(values: List[Any], dstate: DState) = {
		val res = udfEvaluate.invoke(functionClassObject, values: _*) //might need to explicitly cast values to type Object before hand
		res
	}
}

Prototype

UDFunctionProviderImpl.jar needs UDFunctionProvider.jar

MockDaffodil.jar needs UDFunctionProviderImpl.jar & UDFunctionProvider.jar

Diagnostics

We intend to supply the user will at least the following errors/warning

• Warning: No annotation present in function class(es). Class will be ignored
• Error: No User Defined function class with specified name/namespace found
• Error: No evaluate function found in function class object

Testing

• Bad UDFs
  • No provider class
  • No annotation
  • No functionClass list (i.e getFunctionClasses is empty)

Prototype

UDF Jars: HAEMSLConversions.jar and UDFunctionProviderImpl.jar. Both extend UDFunctionProvider.jar.

MockDaffodil.jar contains a MockDaffodil is a Scala app, that also contains a JAVA class that uses ServiceLoader. It needs UDFunctionProviderImpl.jar & UDFunctionProvider.jar