Skip to main content
Log in

Python module

entrypoints

LLM

class max.entrypoints.llm.LLM(settings: Settings, pipeline_config: PipelineConfig)

A high level interface for interacting with LLMs.

generate()

generate(prompts: str | Sequence[str], max_new_tokens: int | None = 100, use_tqdm: bool = True) → list[str]

Generates text completions for the given prompts.

  • Parameters:

    • prompts – The input string or list of strings to generate completions for.
    • max_new_tokens – The maximum number of tokens to generate in the response.
    • use_tqdm – Whether to display a progress bar during generation.
  • Returns:

    A list of generated text completions corresponding to each input prompt.

  • Raises:

    • ValueError – If prompts is empty or contains invalid data.
    • RuntimeError – If the model fails to generate completions.