Mojo struct
RaggedMHAOperand
@register_passable(trivial)
struct RaggedMHAOperand[type_: DType, shape: DimList, stride: DimList]
An implementation for ragged NDBuffer arguments to MHA kernels.
Aliases
type = type_:
Fields
- buffer (
NDBuffer[type_, 3, MutableAnyOrigin, shape, stride]): - cache_row_offsets (
NDBuffer[uint32, 1, MutableAnyOrigin]):
Implemented traits
AnyType,
MHAOperand,
UnknownDestructibility
Methods
__init__
__init__(buffer: NDBuffer[type_, 3, MutableAnyOrigin, shape, stride], cache_row_offsets: NDBuffer[uint32, 1, MutableAnyOrigin, shape, strides]) -> Self
block_paged_ptr
block_paged_ptr[tile_size: Int](self, batch_idx: SIMD[uint32, 1], start_tok_idx: SIMD[uint32, 1], head_idx: SIMD[uint32, 1], head_dim_idx: SIMD[uint32, 1] = __init__[__mlir_type.!pop.int_literal](0)) -> UnsafePointer[SIMD[type_, 1]]
cache_length
cache_length(self, batch_idx: Int) -> Int
max_context_length
max_context_length(self) -> SIMD[uint32, 1]
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!