Python module
sampling
rejection_sampler()
max.pipelines.lib.sampling.rejection_sampler(device, *, seed=0)
rejection_sampler_with_residuals()
max.pipelines.lib.sampling.rejection_sampler_with_residuals(device, *, seed=0, debug=False)
Rejection sampler with residual sampling for speculative decoding.
Computes acceptance ratios for draft tokens, finds first rejection, samples from residual distribution (target - draft), and generates bonus tokens.
token_sampler()
max.pipelines.lib.sampling.token_sampler(sampling_config, device, return_logits=False)
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!