 AlpinDale
|
b8a19ba27f
chore: extend aphrodite metrics logging api
|
7 månader sedan |
|
AlpinDale
|
301ec7c77d
fix: pad slot id in tpu runner
|
7 månader sedan |
|
AlpinDale
|
d0ff3fd59e
fix: tpu sampler output
|
7 månader sedan |
 AlpinDale
|
b6e60143e7
Flashinfer for prefill phase (#580)
|
7 månader sedan |
|
AlpinDale
|
7e66e8f899
fix: only add `Attention.kv_scale` if kv cache quant is enabled
|
7 månader sedan |
|
AlpinDale
|
bbde979ecd
DeepSeek-V2 (#579)
|
7 månader sedan |
|
AlpinDale
|
272c64ab88
chore: allow loading fp8 models with fused qkv/mlp
|
7 månader sedan |
|
AlpinDale
|
772a126c08
chore: simplify fp8 weight loading
|
7 månader sedan |
|
AlpinDale
|
b03b4d4c8c
fix: compute cutlass 3.x epilogues in fp32 instead of 16
|
7 månader sedan |
|
AlpinDale
|
cdff8e89f9
feat: introduce `DraftModelRunner`
|
7 månader sedan |
|
AlpinDale
|
9868bb2290
chore: make it clear that '%' should NOT be in tensor dict keys
|
7 månader sedan |
|
AlpinDale
|
b8650ec51d
fix: better error message for MLPSpeculator
|
7 månader sedan |
|
AlpinDale
|
0886c361f4
feat: OpenVINO CPU backend (#576)
|
7 månader sedan |
|
AlpinDale
|
d63690a0df
chore: add fp8 examples
|
7 månader sedan |
|
AlpinDale
|
b6ff0623a6
chore: clean up branding
|
7 månader sedan |
|
AlpinDale
|
85ef2fe8b1
chore: clean up placeholder symbols
|
7 månader sedan |
|
AlpinDale
|
1852d18326
chore: clean up inference examples
|
7 månader sedan |
|
AlpinDale
|
e1c4cf1d50
chore: organize chat templates
|
7 månader sedan |
|
AlpinDale
|
c0c336aaa3
refactor: registry for processing model inputs; quick_gelu; clip model support
|
7 månader sedan |
|
AlpinDale
|
426a13ab73
fix: pass multi_modal_kwargs to CPU model runner
|
7 månader sedan |
|
AlpinDale
|
bb4da84623
fix: make sure multi modal kwargs can broadcast properly with ring buffer
|
7 månader sedan |
|
AlpinDale
|
d2461161ec
chore: optimize KV cache swapping for TPU
|
7 månader sedan |
|
AlpinDale
|
fad45609b8
chore: remove logical token blocks (turns out they are not needed)
|
7 månader sedan |
|
AlpinDale
|
b3643a7bd7
fix: min_tokens for when there are multiple eos tokens
|
7 månader sedan |
|
AlpinDale
|
51cfadeb29
fix: `MLPSpeculator` handling of `num_speculative_tokens`
|
7 månader sedan |
|
AlpinDale
|
c5d8028668
fix: no need to redefine supports_vision and supports_lora in model class
|
7 månader sedan |
|
AlpinDale
|
b81966c0da
fix: missed phi3v
|
7 månader sedan |
|
AlpinDale
|
56e0b8223c
chore: add base class for LoRA-supported models
|
7 månader sedan |
|
AlpinDale
|
bc5ac9584a
fix: make tensor_dict flattening/unflattening more generic
|
7 månader sedan |
|
AlpinDale
|
dead030abf
fix: cuda graph with MLPSpeculator
|
7 månader sedan |
