Div | ||
---|---|---|
| ||
Software Requirements for: <Project Name>MXNet Java Inference APIs |
Table of Contents | ||||
---|---|---|---|---|
|
Project Introduction
Functional Description
Non-Functional Requirements
User Interface
...
Horizontal Navigation Bar Page | ||||
---|---|---|---|---|
| ||||
Horizontal Navigation Bar Page | ||||
---|---|---|---|---|
| ||||
Milestones
Glossary
...
Problem
There are 2 phases to applying Deep Learning to a ML problem, the first phase is where a neural network is created and trained using training data to generate pre-trained model and In the second phase, this pre-trained model is put to work by running inference(forward-pass) on new data in the customer’s application in production. Model Creation and Training is typically performed by Data Scientists who prefer using Python as a primary language which provides rich set of libraries(numpy, pandas, pillow) etc., to setup the training pipeline. MXNet already has very good support for Python to quickly prototype and develop models.
Inference on the other hand is run and managed by Software Engineers in a production eco-system which is built with tools and frameworks that use Java/Scala as a primary Language.
Inference on a trained model has two different use-cases:
1) Real time or Online Inference - tasks that require immediate feedback, such as fraud detection
2) Batch or Offline Inference - tasks that don't require immediate feedback, these are use-cases where you have massive amounts of data and want to run Inference or pre-compute inference results
Batch Inference is performed on big data platforms such as Spark using Scala or Java.
Real time Inference is typically deployed on popular web frameworks such as Tomcat, Netty, Jetty, etc. which uses Java.
With this project, we want to build Java APIs that are easy to use for Inference and lowers the entry barrier is consuming MXNet for production use-cases.
Goals
Proposed Approach
Alternative Approaches considered
Technical Challenges
Open Questions
Milestones
Glossary