Mojo function
generic_get_paged_cache
generic_get_paged_cache[dtype: DType](blocks: ManagedTensorSlice[IOSpec[True, IO(1)](), static_spec=static_spec], cache_lengths: ManagedTensorSlice[IOSpec[False, IO(1)](), static_spec=static_spec], lookup_table: ManagedTensorSlice[IOSpec[False, IO(1)](), static_spec=static_spec], max_lengths: ManagedTensorSlice[IOSpec[False, IO(1)](), static_spec=static_spec], out result: PagedKVCacheCollection[dtype, KVCacheStaticParams(UInt(static_spec.shape.get[4]()), UInt(static_spec.shape.get[5]())), static_spec.shape.get[3]()])
Returns:
generic_get_paged_cache[dtype: DType, kv_params: KVCacheStaticParams, page_size: Int](blocks: NDBuffer[dtype, 6, origin], cache_lengths: NDBuffer[DType.uint32, 1, origin], lookup_table: NDBuffer[DType.uint32, 2, origin], max_lengths: NDBuffer[DType.uint32, 2, origin], out result: PagedKVCacheCollection[dtype, kv_params, page_size])
Returns:
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!