...
Problem
Taking a MXNet model from training and deploying it in production poses several challenges to the users. Most important problems raised by users are:
...
Code Block |
---|
{
"nodes": [..........]
"arg_nodes": [0, 1, 2, 4, 5],
"node_row_ptr": [0, 1, 2, 3, 4, 5, 6, 7, 8],
"heads": [[7, 0, 0]],
"attrs": {"mxnet_version": ["int", 10301]
"inputs": {"data":[1,3,224,224]},
"outputs" : {"softmax_label":[1,10]
}
} |
...
Possible use cases
Inputs and outputs are dictionary where key => Name of the node and value => shape. Though single input, single output models are more common, a model can be of following variations:
- Single input, single output
- Single input, multiple outputs
- Multiple input with same or different transformation, single output
- Multiple input with same or different transformation, multiple output
Export API - Gluon
- Since this is an update to the most commonly used API - Export in Gluon, we will first introduce the change with a new Export API in gluon.contrib.
Before
Code Block |
---|
|
net.export(path="./my_model", epoch=0) |
After
User Experience
Before
Step 1 - Train and export the model from Gluon
Code Block |
---|
|
# Trained network
net = mx.gluon.model_zoo.vision.resnet18_v1(pretrained=True, ctx=mx.cpu())
# Data transformations applicable during inference
inference_input_transforms = gluon.nn.HybridSequential()
inference_input_transforms.add(transforms.Resize((224, 224)))
inference_input_transforms.add(transforms.ToTensor())
inference_input_transforms.add(transforms.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)))
# Export the model. Cannot export data transformation and input/output signature
net.export(path="./my_model", epoch=0) |
Step 2 - Import the model for inference with Module OR Gluon OR JAVA
Import the model in Gluon
Code Block |
---|
|
# Load the model. Model does not contain transformation and input/output signature
net = SymbolBlock.imports(symbol_file="my_model-symbol.json",
input_names=["data"],
param_file="my_model-0000.params",
ctx=mx.cpu()) |
Import the model in Module API
Code Block |
---|
|
sym, arg_params, aux_params = mx.model.load_checkpoint('my_model', 0)
mod = mx.mod.Module(symbol=sym, context=ctx, label_names=None)
mod.bind(for_training=False, data_shapes=[('data', (1,3,224,224))],
label_shapes=mod._label_shapes)
mod.set_params(arg_params, aux_params, allow_missing=True)
mod.forward(...) |
After
Step 1 - Train and export the model from Gluon
Code Block |
---|
|
# Trained network
net = mx.gluon.model_zoo.vision.resnet18_v1(pretrained=True, ctx=mx.cpu())
# Data transformations applicable during inference
inference_input_transforms = gluon.nn.HybridSequential()
inference_input_transforms.add(transforms.Resize((224, 224)))
inference_input_transforms.add(transforms.ToTensor())
inference_input_transforms.add(transforms.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)))
# Export the model |
Code Block |
---|
|
## net => Hybrid Sequential Network
## input_transforms => Hybrid Sequential Transformation Pipeline
gluon.contrib.utils.export(net, path="./my_model",
epoch=0,
signature={constants.INPUT_DESC:[("data", (1,3,224,224))],
constants.OUTPUT_DESC:[("softmax_label", (1,10))]},
input_transforms={"data":inference_input_transforms},
output_transforms=None) |
Import API - Gluon
...
Step 2 - Import the model for inference with Module OR Gluon OR JAVA
Import the Model in Gluon
Code Block |
---|
|
SymbolBlock.importsgluon.contrib.utils.import(symbol_file="my_model-symbol.json",
input_names=["data"] param_file="my_model-0000.params",
param_file="my_model-0000.params",
load_transforms = True,
ctx = 'cpu') |
OR
Import the Model in Module API
(Supported to create a module for inference only)After
Code Block |
---|
|
gluonmod = mx.contrib.utilsModule.import(
symbol_file = "my_model-symbol.json",
param_file = "my_model-0000.params",
load_transforms = True,
load_transforms = True,
ctx = 'cpu',
ctx = 'cpu') |
Import API - Module
Before
batch_size = 1)
mod.forward(...) |
OR
Import the Model in Java Predictor API
Code Block |
---|
|
List<Context> context = new ArrayList<>();
context.add(Context.cpu());
String modelPathPrefix = "my_model";
Predictor predictor = new Predictor(modelPathPrefix, context, load_transforms=True);
List<NDArray> result = predictor.predictWithNDArray(inputNDArray); |
Import API - Gluon
Before
Code Block |
---|
|
SymbolBlock.imports(symbol_file="my_model-symbol.json",
input_names=["data"],
sym, arg_params, aux_params = mx.model.load_checkpoint('my_model', 0)
mod = mx.mod.Module(symbol=sym, context=ctx, label_names=None)
mod.bind(for_training=False, data_shapes=[('data', (1,3,224,224))],
labelparam_shapes=mod._label_shapes)
mod.set_params(arg_params, aux_params, allow_missing=True)
mod.forward(...)file="my_model-0000.params",
ctx='cpu') |
After
Code Block |
---|
|
gluon.contrib.utils.import(symbol_file="my_model-symbol.json",
param_file="my_model-0000.params",
load_transforms = True,
ctx = 'cpu') |
Import API - Module
Before
After
Supported to create a module for inference only.
...