 AlpinDale
|
30d02d0747
chore: remove peft as a requirement (#1006)
|
2 weeks ago |
|
AlpinDale
|
5c3b94de45
spec decode: move ops.advane_step to flash attention backend (#1005)
|
2 weeks ago |
|
AlpinDale
|
135dfd648b
fix: LoRA support for Cohere and Jamba models (#1004)
|
2 weeks ago |
|
AlpinDale
|
0191c5efd1
tools: fix tool calls to more strictly follow OpenAI format (#1003)
|
2 weeks ago |
|
AlpinDale
|
4d14bd1fe5
vlm: add multi-input support for LLaVA and InternVL models (#1002)
|
2 weeks ago |
|
AlpinDale
|
f561a54a43
core: fix async postprocessor in case of preemption (#1000)
|
2 weeks ago |
|
AlpinDale
|
485d1de42e
fix: hermes tool call chat template (#999)
|
2 weeks ago |
|
AlpinDale
|
cbde3c66a5
quants: improve awq_triton throughput (#998)
|
2 weeks ago |
|
AlpinDale
|
a8ff25679f
chore: use `ray[adag]` dep instead of cuda (#997)
|
2 weeks ago |
|
AlpinDale
|
94a13ad036
fix: gptq_marlin exception on older GPUs (#996)
|
2 weeks ago |
|
AlpinDale
|
548e864404
models: add support for QwenVL (#995)
|
2 weeks ago |
|
AlpinDale
|
145e554a4d
neuron: add 8bit quantization for Neuron (#994)
|
2 weeks ago |
|
AlpinDale
|
313e198557
api: implement OpenAI-compatible tools API for Hermes/Mistral models (#993)
|
2 weeks ago |
|
AlpinDale
|
f644e10449
vlm: enable multimodal inputs for the LLM class (#992)
|
2 weeks ago |
|
AlpinDale
|
46d577f019
vlm: fix siglip layernorm and paligemma weight loading (#991)
|
2 weeks ago |
|
AlpinDale
|
8eb4a3cfd3
vlm: support multiple audios per prompt for Ultravox (#990)
|
2 weeks ago |
|
AlpinDale
|
a50548c0b9
tpu: use XLA rank for persistent cache path (#989)
|
2 weeks ago |
|
AlpinDale
|
231693151b
benchmarks: add `--async-engine` arg to throughput benchmark (#988)
|
2 weeks ago |
|
AlpinDale
|
83af2524f3
quants: add GPTQ and FBGEMM to AphroditeParameters (#987)
|
2 weeks ago |
|
AlpinDale
|
0c56d23ece
tpu: fix outputs by correcting the next_token_ids shape (#986)
|
2 weeks ago |
|
AlpinDale
|
3bb0f07461
chore: rename `task_handler` to `worker` (#985)
|
2 weeks ago |
|
AlpinDale
|
55fa2b9c54
fix: raise exception when accessing logger for disable_log_stats=True case (#984)
|
2 weeks ago |
|
AlpinDale
|
09dab16f82
core: improve async postproc + multi-step performance (#983)
|
2 weeks ago |
|
AlpinDale
|
f56d6b396a
vlm: fallback to SDPA for ViT models on CPU backend (#982)
|
2 weeks ago |
|
AlpinDale
|
df5bf69938
core: slightly improve chunked prefill performance (#981)
|
2 weeks ago |
|
AlpinDale
|
7632f91429
fix: InternLM2 model with Tensor Parallel (#980)
|
2 weeks ago |
|
AlpinDale
|
495435aa7f
tpu: align worker index with node boundary (#979)
|
2 weeks ago |
|
AlpinDale
|
adb6982090
models: add support for IBM Granite (PowerLM) models (#978)
|
2 weeks ago |
 AlpinDale
|
0f72141ca2
api: support multiple input images
|
2 weeks ago |
|
AlpinDale
|
d4e78a428b
fix: crash when cancelling a request with multi-step (#977)
|
2 weeks ago |
