Skip to main content

Mojo struct

Model

Represents a model that's loaded and ready for execution.

Do not instantiate this object directly. Instead, create it with InferenceSession.load(). For example:

var session = engine.InferenceSession()
var model = session.load("bert-base-uncased")
var session = engine.InferenceSession()
var model = session.load("bert-base-uncased")

Then you can run inference by passing your inputs to execute() as a NumPy array, a TensorMap, or one of the other tensor types.

Implemented traits

AnyType, CollectionElement, Copyable, Movable

Methods

__moveinit__

__moveinit__(inout self: Self, owned existing: Self)

Move initializer for model.

Args:

  • existing (Self): Model to move.

__del__

__del__(owned self: Self)

Destructor for Model.

execute

execute(self: Self, inputs: TensorMap) -> TensorMap

Execute model with given inputs.

Args:

  • inputs (TensorMap): A tensor map with input names as keys and inputs as values.

Returns:

A TensorMap with output names as keys.

execute(self: Self, inputs: PythonObject) -> TensorMap

Execute model with given inputs.

Args:

  • inputs (PythonObject): Inputs as a Python object, which must be a dictionary with string keys (matching input names) and NumPy array values.

Returns:

A TensorMap with output names as keys.

execute(self: Self, *inputs: NamedTensor) -> TensorMap

Execute model with given inputs.

Args:

  • *inputs (NamedTensor): A variadic list of NamedTensor values.

Returns:

A TensorMap with output names as keys.

execute(self: Self, *inputs: Tuple[StringLiteral, EngineNumpyView]) -> TensorMap

Execute model with given inputs.

Args:

  • *inputs (Tuple[StringLiteral, EngineNumpyView]): A variadic list of tuples with first element of tuple is input name and second element is non owning view of a Numpy array.

Returns:

A TensorMap with output names as keys.

execute[type: DType](self: Self, name: String, input: Tensor[type]) -> TensorMap

Execute model with given input.

Parameters:

  • type (DType): DType of input tensor.

Args:

  • name (String): Name of the input tensor.
  • input (Tensor[type]): Input tensor to the model.

Returns:

A TensorMap with output names as keys.

execute(self: Self, name: String, inout input: PythonObject) -> TensorMap

Execute model with given input.

Args:

  • name (String): Name of the input tensor.
  • input (PythonObject): Input to the model as numpy array.

Returns:

A TensorMap with output names as keys.

execute[type1: DType, type2: DType](self: Self, name1: String, input1: Tensor[type1], name2: String, input2: Tensor[type2]) -> TensorMap

Execute model with given inputs.

Parameters:

  • type1 (DType): DType of first input tensor.
  • type2 (DType): DType of second input tensor.

Args:

  • name1 (String): Name of the first input tensor.
  • input1 (Tensor[type1]): First Input tensor to the model.
  • name2 (String): Name of the second input tensor.
  • input2 (Tensor[type2]): Second Input tensor to the model.

Returns:

A TensorMap with output names as keys.

execute(self: Self, name1: String, inout input1: PythonObject, name2: String, inout input2: PythonObject) -> TensorMap

Execute model with given inputs.

Args:

  • name1 (String): Name of the first input tensor.
  • input1 (PythonObject): First Input to the model as numpy array.
  • name2 (String): Name of the second input tensor.
  • input2 (PythonObject): Second Input to the model as numpy array.

Returns:

A TensorMap with output names as keys.

execute[type1: DType, type2: DType, type3: DType](self: Self, name1: String, input1: Tensor[type1], name2: String, input2: Tensor[type2], name3: String, input3: Tensor[type3]) -> TensorMap

Execute model with given inputs.

Parameters:

  • type1 (DType): DType of first input tensor.
  • type2 (DType): DType of second input tensor.
  • type3 (DType): DType of third input tensor.

Args:

  • name1 (String): Name of the first input tensor.
  • input1 (Tensor[type1]): First Input tensor to the model.
  • name2 (String): Name of the second input tensor.
  • input2 (Tensor[type2]): Second Input tensor to the model.
  • name3 (String): Name of the third input tensor.
  • input3 (Tensor[type3]): Third Input tensor to the model.

Returns:

A TensorMap with output names as keys.

execute(self: Self, name1: String, inout input1: PythonObject, name2: String, inout input2: PythonObject, name3: String, inout input3: PythonObject) -> TensorMap

Execute model with given inputs.

Args:

  • name1 (String): Name of the first input tensor.
  • input1 (PythonObject): First Input to the model as numpy array.
  • name2 (String): Name of the second input tensor.
  • input2 (PythonObject): Second Input to the model as numpy array.
  • name3 (String): Name of the third input tensor.
  • input3 (PythonObject): Third Input to the model as numpy array.

Returns:

A TensorMap with output names as keys.

execute(self: Self, owned *inputs: AnyMemory) -> List[AnyMemory, 0]

Execute model with the given inputs.

Arguments: inputs: Inputs can either be tensors or mojo objects for opaque types defined in the graph, which may be located on any Device. If there are both tensor and opaque inputs, all tensor inputs should come before opaque inputs in graph. If inputs are tensors, API will automatically copy the input tensors to the Device set in the InferenceSession's SessionConfig.

Returns:

A list of outputs which can either be output tensors, which are located on the Device set in the InferenceSession's SessionConfig, or opaque mojo types as defined in the graph.

num_model_inputs

num_model_inputs(self: Self) -> Int

Gets the number of inputs of the model.

Returns:

Number of inputs of model.

get_model_input_names

get_model_input_names(self: Self) -> List[String, 0]

Gets the names of model inputs.

Returns:

Input names of the model.

num_model_outputs

num_model_outputs(self: Self) -> Int

Gets the number of outputs of the model.

Returns:

Number of model outputs.

get_model_output_names

get_model_output_names(self: Self) -> List[String, 0]

Gets the names of model outputs.

Returns:

Output names of the model.

get_model_input_metadata

get_model_input_metadata(self: Self) -> List[EngineTensorSpec, 0]

Get metadata about the model's input tensors, as a list of EngineTensorSpec objects.

Returns:

Metadata list of the model's input tensors.

get_model_output_metadata

get_model_output_metadata(self: Self) -> List[EngineTensorSpec, 0]

Get metadata about the model's output tensors, as a list of EngineTensorSpec objects.

Returns:

Metadata list of the model's output tensors.

export_compiled_model

export_compiled_model(self: Self, path: String)

Exports a compiled model as a MEF to a given path. Args: path: The path of the MEF file to export.