 AlpinDale
|
60ca1e1e5e
feat: add ngram prompt lookup decoding for speculative decoding (#438)
|
vor 9 Monaten |
|
AlpinDale
|
d8c4193704
feat: Speculative Decoding using a draft model (#432)
|
vor 9 Monaten |
|
AlpinDale
|
76f36af704
feat: LM Format Enforcer support (#428)
|
vor 9 Monaten |
|
AlpinDale
|
bd0ddf1cfe
feat: EETQ quantization (#408)
|
vor 10 Monaten |
|
AlpinDale
|
4d33ce60da
feat: Triton flash attention backend for ROCm (#407)
|
vor 10 Monaten |
|
AlpinDale
|
071269e406
feat: FP8 E4M3 KV Cache (#405)
|
vor 10 Monaten |
 AlpinDale
|
6f00203041
refactor scheduler for chunked prefill, remove reorder policy for now
|
vor 10 Monaten |
|
AlpinDale
|
9aaeb5d349
add speculative config and arg for later
|
vor 10 Monaten |
|
AlpinDale
|
a304f76d89
feat: Intel CPU support (#403)
|
vor 10 Monaten |
|
AlpinDale
|
fa083286e3
Speculative Decoding Part 4: Lookahead scheduling (#402)
|
vor 10 Monaten |
|
AlpinDale
|
f845a661dd
Chunked Prefill Part 2: data update
|
vor 10 Monaten |
|
AlpinDale
|
eff5eb16c5
ruff
|
vor 10 Monaten |
|
AlpinDale
|
753f6dc51b
add v2 block manager
|
vor 10 Monaten |
|
AlpinDale
|
7b9c08afae
vision model support
|
vor 10 Monaten |
|
AlpinDale
|
b738554558
add reorder scheduler policy
|
vor 10 Monaten |
|
AlpinDale
|
1ba9ff78cd
add scheduler delay factor
|
vor 10 Monaten |
|
AlpinDale
|
78d66f16d1
Chunked Prefill Part 1 (#384)
|
vor 10 Monaten |
|
AlpinDale
|
feb5840f2a
feat: async tokenization (#374)
|
vor 10 Monaten |
|
AlpinDale
|
f8dfac6372
chore: attention refactor and upstream sync apr01 (#365)
|
vor 10 Monaten |
|
AlpinDale
|
c41462cfcd
feat: exllamav2 quantization (#305)
|
vor 11 Monaten |
|
AlpinDale
|
9810daa699
feat: INT8 KV Cache (#298)
|
vor 11 Monaten |
|
AlpinDale
|
e0c35bb353
feat: bitsandbytes and `--load-in{4,8}bit` support (#294)
|
vor 11 Monaten |
|
AlpinDale
|
705821a7fe
feat: AQLM quantization support (#293)
|
vor 11 Monaten |
|
AlpinDale
|
ac82b67f75
feat: naive context shift and various QoL changes (#289)
|
vor 11 Monaten |
|
AlpinDale
|
72229a94da
feat: better marlin kernels (#285)
|
vor 11 Monaten |
|
AlpinDale
|
657aec0cbd
refactor: OpenAI endpoint (#261)
|
vor 11 Monaten |
|
AlpinDale
|
4d04ade9ef
feat: fine-grained seeds (#279)
|
vor 11 Monaten |
|
AlpinDale
|
ea0f57b233
feat: allow further support for non-cuda devices (#247)
|
vor 1 Jahr |
|
AlpinDale
|
c3a221eb02
feat: GGUF, QuIP#, and Marlin support (#228)
|
vor 1 Jahr |
|
AlpinDale
|
31c95011a6
feat: FP8 E5M2 KV Cache (#226)
|
vor 1 Jahr |
