Mojo module

encodings

Implementations of quantization encodings.

Aliases

• ​K_SCALE_SIZE = IntLiteral(12): Size of superblock scales and mins, in bytes.
• ​QK_K = IntLiteral(256): Size of superblock quantized elements, in bytes.

Structs

• ​BFloat16Encoding: The bfloat16 quantization encoding.
• ​Float32Encoding: The float32 quantization encoding.
• ​Q4_0Encoding: The Q4_0 quantization encoding.
• ​Q4_KEncoding: The Q4_K quantization encoding.
• ​Q5_KEncoding: The Q5_K quantization encoding.
• ​Q6_KEncoding: The Q6_K quantization encoding.

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit