Skip to main content

/

Docs

Nightly
v24.5
Log in

Context

#include "max/c/context.h"
#include "max/c/context.h"

Functions​

M_newRuntimeConfig()​

M_RuntimeConfig *M_newRuntimeConfig()

Creates a new runtime config.

This configures runtime details such as the number of threads and log level.

By default, the config object’s number of threads will be set to 0, which is internally used to refer to the number of physical processors in the first socket in the system. You can change this with M_setNumThreads().

You need this as an argument for M_newRuntimeContext().

  • Returns:

    A pointer to the new runtime config. You are responsible for the memory associated with the pointer returned. You can deallocate the memory by calling M_freeRuntimeConfig().

M_setArtifactPath()​

void M_setArtifactPath(M_RuntimeConfig *runtimeConfig, const char *path, M_Status *status)

Sets the location where a runtime stores its artifacts.

These artifacts are required for compilation and execution of models. If no artifact path is set, the runtime defaults to using a temporary directory.

  • Parameters:

    • runtimeConfig – The runtime config.
    • path – The artifacts path.
    • status – The status object for reporting errors.

M_setNumThreads()​

void M_setNumThreads(M_RuntimeConfig *config, size_t numThreads)

Sets the number of threads in a runtime’s threadpool.

  • Parameters:

    • config – The runtime config.
    • numThreads – The number of threads.

M_setAllocatorType()​

void M_setAllocatorType(M_RuntimeConfig *config, M_AllocatorType allocatorType)

Sets the memory allocator used for tensor allocations.

  • Parameters:

    • config – The runtime config.
    • allocatorType – An identifier for the type of allocator to use. Currently must be kCaching or kSystem. kCaching uses an allocator that trades-off performance for memory usage by not freeing memory immediately to the system. kSystem on the other hand frees memory immediately to the system and may not be as performant in some cases. The default is kCaching.

M_setCPUAffinity()​

void M_setCPUAffinity(M_RuntimeConfig *config, bool cpuAffinity)

Sets whether CPU affinity is enabled.

  • Parameters:

    • config – The runtime config.
    • cpuAffinity – The new CPU affinity setting.

M_getNumThreads()​

size_t M_getNumThreads(M_RuntimeConfig *config)

Gets the number of threads in a runtime’s threadpool.

  • Parameters:

    config – The runtime config.

  • Returns:

    The number of threads in the the runtime’s threadpool. Otherwise, 0 if M_setNumThreads() has not been called.

M_getCPUAffinity()​

bool M_getCPUAffinity(M_RuntimeConfig *config)

Gets the current CPU affinity setting. Note that this does not guarantee that any CPU affinity will be set, however if this is false then it is guarantee that no CPU affinity will be set.

  • Parameters:

    config – The runtime config.

  • Returns:

    The current CPU affinity setting.

M_enableCrashLog()​

void M_enableCrashLog(M_RuntimeConfig *config, const char *crashDir)

Enables crash logging and sets the location where crash dumps are stored. Note that this will install signal handlers to do so; ensure that this method is called last to unwind to previously registered handlers.

  • Parameters:

    crashDir – The crash dump directory.

M_freeRuntimeConfig()​

void M_freeRuntimeConfig(M_RuntimeConfig *config)

Deallocates the memory for a runtime config. No-op if config is NULL.

  • Parameters:

    config – The runtime config.

M_newRuntimeContext()​

M_RuntimeContext *M_newRuntimeContext(const M_RuntimeConfig *config, M_Status *status)

Creates a runtime context.

The context is an application-level object that sets up various resources such as threadpool and allocators during inference. You need this before you can call M_compileModel().

It’s expected that there’s only one runtime context active in an inference session at a time. We recommended you create one context and use it throughout your application.

For example:

M_Status *status = M_newStatus();
M_RuntimeConfig *runtimeConfig = M_newRuntimeConfig();
M_RuntimeContext *context = M_newRuntimeContext(runtimeConfig, status);
if (M_isError(status)) {
  logError(M_getError(status));
  return EXIT_FAILURE;
}
M_Status *status = M_newStatus();
M_RuntimeConfig *runtimeConfig = M_newRuntimeConfig();
M_RuntimeContext *context = M_newRuntimeContext(runtimeConfig, status);
if (M_isError(status)) {
  logError(M_getError(status));
  return EXIT_FAILURE;
}

  • Parameters:

    • config – The runtime config, from M_newRuntimeConfig().
    • status – The status object for reporting errors. It is filled with an error message if construction of the runtime context fails.

  • Returns:

    A pointer to the runtime context object. On success, this is a valid pointer. On failure, this is a NULL pointer with an error message in the status. You are responsible for the memory associated with the pointer returned. You can deallocate the memory by calling M_freeRuntimeContext().

M_freeRuntimeContext()​

void M_freeRuntimeContext(M_RuntimeContext *context)

Deallocates the memory for a runtime context. No-op if context is NULL.

  • Parameters:

    context – The runtime context.

M_setDebugPrintOptions()​

void M_setDebugPrintOptions(M_RuntimeContext *context, M_ResultOutputStyle style, unsigned int precision, const char *directory)

Set the options for debugging printing of tensors when executing a model.

  • Parameters:

    • context – The runtime context.
    • style – The way the data will be printed.
    • precision – The floating point print out precision.
    • directory – The directory to store binary output.