Page History

...

• With this enhanced custom operator profiling, we also want to get rid of profiling "Custom" and “Dummy_Wait” entirely. This is done by adding a check in ProfileOperator in profiler.h.
• Notice that because we are adding a function call to GenerateDisplayName() in PushAsync(), we are risking adding an overhead to every operator call (we need to get thread id and and the function has a lock). However in practice, because this function is short and has early return checks, this overhead is small enough to forgive. On my machine (2017 MacBook Pro 13’ i7), on average, for regular operator calls, this overhead is less than 1 micro second (it appears as 0). And for sub-operator calls, the overhead is always < 10 micro seconds and averages to < 5 micro seconds. This is to be compared to ~150 micro seconds taken by executing NDArray plus scalar on a 100*100 matrix. Notice this relative larger overhead will only happen to sub-operators of custom operators.
• Currently CustomOperator has its own thread pool. In the future this may change so that custom operator calls will use engine worker threads directly. In that prospect, the proposed mapping from thread_id to custom operator names would continue to work.

...