| .. |
|
block
|
d8c4193704
feat: Speculative Decoding using a draft model (#432)
|
há 8 meses atrás |
|
__init__.py
|
ac1d46a2ec
feat: begin work on the engine
|
há 1 ano atrás |
|
block_manager_v1.py
|
f52aa64fe6
use the get_len() method instead of manual len calculation
|
há 9 meses atrás |
|
block_manager_v2.py
|
fa083286e3
Speculative Decoding Part 4: Lookahead scheduling (#402)
|
há 9 meses atrás |
|
evictor.py
|
375f24ccca
fix: optimize context shift performance (#380)
|
há 9 meses atrás |
|
interfaces.py
|
fa083286e3
Speculative Decoding Part 4: Lookahead scheduling (#402)
|
há 9 meses atrás |
|
policy.py
|
6f00203041
refactor scheduler for chunked prefill, remove reorder policy for now
|
há 9 meses atrás |
|
scheduler.py
|
c577c31aaa
feat: tree attention
|
há 8 meses atrás |
AlpinDale
c577c31aaa
feat: tree attention