Page History

...

import mxnet.ndarray as nd
from mxnet import autograd

x = nd.array([1, 2, 3])
x.attach_grad()
with autograd.record():
    y = nd.sin(x)
    # y_grad is first order gradient of y and should be cos(x)
    y_grad = autograd.grad(y, x, create_graph=True, retain_graph=True)[0]
# this call should calculate the second order of y w.r.t x which should be -sin(x)
y_grad.backward()
print(x.grad) # Should be -sin(x)

Goals/Usecases

...

This project will implement a set of operators to support adaptive learning rate optimization proposed by http://proceedings.mlr.press/v70/franceschi17a.html. The following operators will be supported in this project

Open Questions

TBD

Proposed Approach

MXNet Java Inference API#ProposedApproach

MXNet Java Inference API#ClassDiagram

MXNet Java Inference API#SequenceDiagramThe reason that many operators currently do not support second or higher order gradient is because that 

Addition of New APIs

Backward compatibility

...