Versions Compared

Old Version 24

changes.mady.by.user Wencong Liu

Saved on

compared with

New Version 25

changes.mady.by.user Wencong Liu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

1. In the implementation of mapPartition API, make operator do not cache records. The UDFExecutionThread must execute synchronously with TaskMainThread to get the next record and process it in MapPartitionFunction.

We choose not to use this approach because it will result in inefficient execution due to frequent thread context switching between TaskMainThread and UDFExecutionThread. Caching records enables concurrent execution of the two threads and reduces the frequency of context switching.

...