Mojo module

mha_fa3_utils

Aliases

QTMATile

alias QTMATile[dtype: DType, swizzle_mode: TensorMapSwizzle, *, BM: Int, depth: Int, group: Int, decoding: Bool] = TMATensorTile[dtype, tile_layout_k_major[dtype, max(group, 8) if decoding else BM, 64 if decoding else depth, swizzle_mode](), _tma_desc_tile_layout[dtype, 2, IndexList[2, DType.int64](max(group, 8) if decoding else BM, 64 if decoding else depth, Tuple[]()), swizzle_mode=swizzle_mode]()]

Parameters

• ​dtype (DType):
• ​swizzle_mode (TensorMapSwizzle):
• ​BM (Int):
• ​depth (Int):
• ​group (Int):
• ​decoding (Bool):

Structs

• ​MHAPosition: Position of the MHA-kernel. When decoding=False, q_head_stride == q_num_heads. When decoding=True, q_head_stride == 1.
• ​NonNullPointer:
• ​NullPointer:
• ​Pack:

Traits

• ​OptionalPointer:

Functions

• ​output_reg_to_smem:
• ​produce:
• ​q_out_tma:

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit