Python module
entrypoints
LLM
class max.entrypoints.llm.LLM(settings: Settings, pipeline_config: PipelineConfig)
A high level interface for interacting with LLMs.
generate()
generate(prompts: str | Sequence[str], max_new_tokens: int | None = 100, use_tqdm: bool = True) → list[str]
Generates text completions for the given prompts.
-
Parameters:
- prompts – The input string or list of strings to generate completions for.
- max_new_tokens – The maximum number of tokens to generate in the response.
- use_tqdm – Whether to display a progress bar during generation.
-
Returns:
A list of generated text completions corresponding to each input prompt.
-
Raises:
- ValueError – If prompts is empty or contains invalid data.
- RuntimeError – If the model fails to generate completions.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!