Page History

Problem Description

Currently, Clojure's NDArray and Symbol APIs are generated based on reflection of Java classes. These classes contain function definitions that are generated by the Scala package based on the C/C++ API using JNI.

...

Scala package also provides a new typesafe API that can potentially be used to generate new ndarray-api and symbol-api packages for Clojure. This was recently attempted in NDArray/Symbol API PR. However, such an approach quickly becomes cumbersome especially when applied to functions with lot of arguments (e.g. convolution or rnn). For example, the signature of convolution function using the new approach becomes (convolution ndarray ndarray-1 ndarray-2 shape option option-1 option-2 num option-3 option-4 option-5 option-6 option-7 option-8 option-9). Although this approach (better) clarifies the arguments to the function it has many of the same problems as before.

Proposal

Unlike Scala's focus on types, the ideal clojure APIs should embrace the data structures available within the language, allow definition of specs and describe the arguments better (e.g. using better names, documentation).

Let's take the simple ndarray activation function as an example to see what the ideal state would look like:

(s/def ::ndarray ...#(instance? NDArray %))
(s/def ::act-type #{"relu" "sigmoid" ...})
(s/fdef ::activation ...)

(defn activation

...

   ([data weight bias kernel num-filter
 {:keys [stride dilate pad num-group workspace
  no-bias cudnn-tune cudnn-off layout out]
 :as opts}]

     ...)

   ([data weight bias kernel num-filter]
 (convolution data weight bias kernel num-filter {})))

...

Although this function is called the same way today, the main difference is that the API is more explicit.

Scala => Clojure Translation

We should also encourage the usage of Clojure data structures as function arguments:

Potential Approach(es)

We can potentially use the Scala package's GeneratorBase to create the functions described above. The current challenge is that the generator cannot be called directly from within Clojure and hence we have to rely on reflection instead.

...