 AlpinDale
|
f644e10449
vlm: enable multimodal inputs for the LLM class (#992)
|
1 month ago |
|
AlpinDale
|
46d577f019
vlm: fix siglip layernorm and paligemma weight loading (#991)
|
1 month ago |
|
AlpinDale
|
8eb4a3cfd3
vlm: support multiple audios per prompt for Ultravox (#990)
|
1 month ago |
|
AlpinDale
|
a50548c0b9
tpu: use XLA rank for persistent cache path (#989)
|
1 month ago |
|
AlpinDale
|
231693151b
benchmarks: add `--async-engine` arg to throughput benchmark (#988)
|
1 month ago |
|
AlpinDale
|
83af2524f3
quants: add GPTQ and FBGEMM to AphroditeParameters (#987)
|
1 month ago |
|
AlpinDale
|
0c56d23ece
tpu: fix outputs by correcting the next_token_ids shape (#986)
|
1 month ago |
|
AlpinDale
|
3bb0f07461
chore: rename `task_handler` to `worker` (#985)
|
1 month ago |
|
AlpinDale
|
55fa2b9c54
fix: raise exception when accessing logger for disable_log_stats=True case (#984)
|
1 month ago |
|
AlpinDale
|
09dab16f82
core: improve async postproc + multi-step performance (#983)
|
1 month ago |
|
AlpinDale
|
f56d6b396a
vlm: fallback to SDPA for ViT models on CPU backend (#982)
|
1 month ago |
|
AlpinDale
|
df5bf69938
core: slightly improve chunked prefill performance (#981)
|
1 month ago |
|
AlpinDale
|
7632f91429
fix: InternLM2 model with Tensor Parallel (#980)
|
1 month ago |
|
AlpinDale
|
495435aa7f
tpu: align worker index with node boundary (#979)
|
1 month ago |
|
AlpinDale
|
adb6982090
models: add support for IBM Granite (PowerLM) models (#978)
|
1 month ago |
 AlpinDale
|
0f72141ca2
api: support multiple input images
|
1 month ago |
|
AlpinDale
|
d4e78a428b
fix: crash when cancelling a request with multi-step (#977)
|
1 month ago |
|
AlpinDale
|
3e4e7665a7
fix: modelscope for VLMs (#976)
|
1 month ago |
|
AlpinDale
|
201db10f02
models: add support for Phi3 MoE
|
1 month ago |
|
AlpinDale
|
032974a28a
tpu: fix TPU type api (#975)
|
1 month ago |
|
AlpinDale
|
510ae5b949
core: fix chunked prefill not being enabled by default for long contexts (#974)
|
1 month ago |
|
AlpinDale
|
b3f6eeb1d2
vlm: increase the default `max_num_batched_tokens` for multimodal models (#973)
|
1 month ago |
|
AlpinDale
|
7eeee771f2
tests: update internvl test for #971 (#972)
|
1 month ago |
|
AlpinDale
|
b4a1e2fd02
vlm: add tensor parallel support for vision transformer models (#971)
|
1 month ago |
|
AlpinDale
|
61103b92d4
tpu: support single and multi-host TPUs on GKE and RayServe (#970)
|
1 month ago |
|
AlpinDale
|
b26a014b12
fix: prometheus.yaml path in monitoring example (#969)
|
1 month ago |
|
AlpinDale
|
5bec8fbb1b
tpu: add support for async postprocessing (#968)
|
1 month ago |
|
AlpinDale
|
a8bdd488b9
distributed: support pipeline parallelism for internvl and internlm2 (#965)
|
1 month ago |
|
AlpinDale
|
cbd51a208a
ci: bump to 0.6.5 (#964)
|
1 month ago |
|
AlpinDale
|
0dfa6b60ec
core: support logprobs with multi-step scheduling (#963)
|
1 month ago |
