Skip to main content

Python class

BackgroundRecorder

BackgroundRecorder

class max.diagnostics.gpu.BackgroundRecorder

Asynchronous GPU metrics collection and data export capabilities.

The BackgroundRecorder enables continuous monitoring of GPU performance metrics without blocking the main application thread. It automatically samples GPU statistics at one-second intervals in a separate process, making it ideal for profiling long-running inference sessions or training workloads.

When used as a context manager, the recorder starts background collection upon entry and stops collection upon exit. The collected statistics are then available through the stats property as a time-series of GPU measurements.

from max.diagnostics.gpu import BackgroundRecorder

with BackgroundRecorder() as recorder:
    # Run your GPU workload here
    run_inference_session()

for i, snapshot in enumerate(recorder.stats):
    print(f"Sample {i}: {len(snapshot)} GPUs detected")
    for gpu_id, gpu_stats in snapshot.items():
        print(f"  {gpu_id}: {gpu_stats.memory.used_bytes} bytes used")

stats

property stats: list[dict[str, GPUStats]]

Time-series of GPU statistics collected during background recording.

Returns:

A list of dictionaries, where each dictionary represents GPU statistics at a specific point in time. Each dictionary maps GPU identifiers to their corresponding GPUStats objects.

Raises:

RuntimeError – If accessed before the recorder context has exited.