PyTorch config
#include "max/c/pytorch/config.h"
#include "max/c/pytorch/config.h"
Functions
M_newTorchInputSpec()
M_TorchInputSpec *M_newTorchInputSpec(const int64_t *shape, const char *const *dimNames, int64_t rankSize, M_Dtype type, M_Status *status)
Creates TorchScript input specification.
You need the M_TorchInputSpec
object returned here as an argument for M_setTorchInputSpecs()
.
If the model supports an input with dynamic shapes, use M_getDynamicDimensionValue()
for that dimension size.
For example:
// Static input shape:
int64_t shape1[] = {100, 200};
M_TorchInputSpec *inputSpec1 = M_newTorchInputSpec(/*shape=*/shape1,
/*dimNames=*/NULL,
/*rankSize=*/2,
/*dtype=*/M_INT32,
/*status=*/status);
// Dynamic input shape:
int64_t shape2[] = {100, 200, M_getDynamicDimensionValue()};
M_TorchInputSpec *inputSpec2 = M_newTorchInputSpec(/*shape=*/shape2,
/*dimNames=*/NULL,
/*rankSize=*/3,
/*dtype=*/M_INT32,
/*status=*/status);
M_TorchInputSpec *inputSpecs[2] = {inputSpec1, inputSpec2};
M_setTorchInputSpecs(compileConfig, inputSpecs, 2);
M_AsyncCompiledModel *compiledModel = M_compileModel(context,
&compileConfig,
status);
// Static input shape:
int64_t shape1[] = {100, 200};
M_TorchInputSpec *inputSpec1 = M_newTorchInputSpec(/*shape=*/shape1,
/*dimNames=*/NULL,
/*rankSize=*/2,
/*dtype=*/M_INT32,
/*status=*/status);
// Dynamic input shape:
int64_t shape2[] = {100, 200, M_getDynamicDimensionValue()};
M_TorchInputSpec *inputSpec2 = M_newTorchInputSpec(/*shape=*/shape2,
/*dimNames=*/NULL,
/*rankSize=*/3,
/*dtype=*/M_INT32,
/*status=*/status);
M_TorchInputSpec *inputSpecs[2] = {inputSpec1, inputSpec2};
M_setTorchInputSpecs(compileConfig, inputSpecs, 2);
M_AsyncCompiledModel *compiledModel = M_compileModel(context,
&compileConfig,
status);
Note: When storing data in memory, we always use a diminishing stride size. That is, earlier dimensions in the shape have larger strides than later dimensions. For example, a C array declared as int arr[1][2][3]
would have a shape specified as {1, 2, 3}
.
Input specs may also optionally name their dimensions. If you wish to name the dimensions, pass an array of size rankSize
of NUL-terminated strings in the dimNames
parameter. A dimension’s name may be NULL
if you do not want to name that dimension. Dimension names may only contain alphanumeric characters and underscores, and the initial character of a dimension name must not be numeric.
-
Parameters:
- shape – The input tensor shape, if rank is static. Otherwise, use
NULL
, if the shape is fully dynamic. - dimNames – Optional array of optional dimension names. Array may be left
NULL
to leave all dimensions unnamed, and elements may be leftNULL
to leave individual dimensions unnamed. - rankSize – The input tensor rank, if rank is static. Otherwise, use
M_getDynamicRankValue()
, if the rank is unknown (the shape is fully dynamic). Note that the rank can still be static even when some dimension sizes are dynamic (such as when only the batch size is dynamic); in that case, useM_getDynamicDimensionValue()
for that dimension size (in theshape
). - dtype – The datatype for the input.
- status – Status used to report errors.
- shape – The input tensor shape, if rank is static. Otherwise, use
-
Returns:
A pointer to the input spec, or
NULL
in case of failure. You are responsible for the memory associated with the pointer returned. The memory can be deallocated by callingM_freeTorchInputSpec()
.
M_setTorchInputSpecs()
void M_setTorchInputSpecs(M_CompileConfig *config, M_TorchInputSpec **inputSpecs, size_t inputSpecsSize)
Sets the input specifications for a TorchScript model.
You must call this before you compile a TorchScript model with M_compileModel()
, in order to specify the input specs. (This is not needed to compile a TensorFlow SavedModel or ONNX model.)
-
Parameters:
- config – The compilation configuration for your model.
- inputSpecs – The input specifications, including the shape, rank, and type for each input tensor. These specs are copied into the configuration, so it’s safe to release the
M_TensorSpec
array after this function returns. - inputSpecsSize – The number of input specifications to set.
M_setTorchLibraryPath()
void M_setTorchLibraryPath(M_CompileConfig *config, const char *path)
Sets the torch library path for a TorchScript model.
-
Parameters:
- config – The compilation configuration for your model.
- path – The torch library path.
M_setTorchMetadataLibraryPath()
void M_setTorchMetadataLibraryPath(M_CompileConfig *config, const char *path)
Sets the torch metadata library path for a TorchScript model.
-
Parameters:
- config – The compilation configuration for your model.
- path – The torch metadata library path.
M_setTorchMetadataLibraryPtr()
void M_setTorchMetadataLibraryPtr(M_CompileConfig *config, void *ptr)
Sets the torch metadata library pointer for a TorchScript model.
-
Parameters:
- config – The compilation configuration for your model.
- ptr – The torch metadata library path.
M_freeTorchInputSpec()
void M_freeTorchInputSpec(M_TorchInputSpec *compileSpec)
Deallocates the memory for the input spec. No-op if spec
is NULL.
-
Parameters:
spec – The input spec to deallocate.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!