...

2 Execution on Vectorized Engine

2.1 Fixed-length output

First of all, a new Java function call will be executed in Doris query engine and BE will create or reuse a JVM to call the real Java UDF. To isolate different UDF instances, we use different class loader to load UDF.

Secondly, it’s a better way to implement JNI call using strided execution than row-by-row execution. As mentioned in paper [1], this leads to a much better performance since the JNI overhead is amortized by all rows in input columns.

Notably, user must follow certain rules when creating UDFs. For example, UDF class must have evaluate  method and it's must be public and non-static. These rules ensure we can invoke UDF correctly.

2.1 Fixed-length output

To use strided execution mode, a basic idea is passing addresses which point to input buffer and output buffer directly. This can help us to avoid unnecessary data copies. Input buffer and output buffer are both jvm JVM off-heap memory and fortunately, Java provides API to manipulate off-heap memory for us.

So overall execution mode is illustrated in figure 1:

Image RemovedImage Added

figure 1

Step 1, allocate output buffer for UDF.

...

For fixed-length type input and output, this is a standardized process because each buffer size is fixed. But for variable-length output type, above steps are no longer applicable. For variable-length output, we always allocate a an initial buffer size in Step 1, and jump out of Step 3 when size of results is bigger than initial buffer allocated in Step 1. When it happens, we repeat Step 1 ~3 again to allocate a new buffer and continue to execute UDF for remain rows. To do this, we should maintain some states to ensure correctness. So this process is illustrated as figure 2:

...

In some cases, above process works well. Unfortunately, it is not a perfect choice for each case. Assume that each input column can fill the cache, when we access each row in each column in Step 3, every access will incur a cache miss. So in the worst case, Step 3 will incur numRows * numColumns cache misses which is a performance disaster.

So we should copy data properly to improve data locality as illustrated in figure 3.

Image RemovedImage Added

figure 3

Notice that, in step 1, we use an additional operation to convert column-cased input to row-based.

Although this method can avoid cache missed as much as possible, the most critical issue is when we should use a row-based input to call a UDF. This is a trade-off between data copies and cache misses. This strategy is still unclear and we need to discuss later.

Scheduling

I have a preliminary plan to support Java UDF in Doris.

...