 AlpinDale
|
6671e3a162
feat: add CPU offloading support (#598)
|
hace 5 meses |
 AlpinDale
|
22305c91e9
refactor _prepare_model_input_tensor and attn metadata builder for most backends
|
hace 5 meses |
|
AlpinDale
|
5289c14b24
feat: Asymmetric Tensor Parallel (#594)
|
hace 5 meses |
|
AlpinDale
|
99680b2d23
feat: soft prompts (#589)
|
hace 5 meses |
|
AlpinDale
|
c11a8bdaad
fix: calculate max number of multi-modal tokens automatically
|
hace 5 meses |
|
AlpinDale
|
151d782233
fix: attention softcapping for flashinfer
|
hace 5 meses |
|
AlpinDale
|
4f7d212b70
feat: remove vision language config
|
hace 5 meses |
|
AlpinDale
|
4599c98f99
feat: dynamic image size support for VLMs
|
hace 5 meses |
|
AlpinDale
|
5be90c3859
Mamba infrastrucuture support (#586)
|
hace 5 meses |
|
AlpinDale
|
ae04f57ec1
feat: Pipeline Parallel support (#581)
|
hace 5 meses |
|
AlpinDale
|
3a0fdf7b9b
chore: remove `image_input_type` from VLM config
|
hace 5 meses |
|
AlpinDale
|
b6e60143e7
Flashinfer for prefill phase (#580)
|
hace 5 meses |
|
AlpinDale
|
cdff8e89f9
feat: introduce `DraftModelRunner`
|
hace 5 meses |
|
AlpinDale
|
c0c336aaa3
refactor: registry for processing model inputs; quick_gelu; clip model support
|
hace 5 meses |
|
AlpinDale
|
56e0b8223c
chore: add base class for LoRA-supported models
|
hace 5 meses |
|
AlpinDale
|
dead030abf
fix: cuda graph with MLPSpeculator
|
hace 5 meses |
|
AlpinDale
|
405bb74612
Control plane comms refactor (#573)
|
hace 5 meses |
|
AlpinDale
|
25feb1d592
chore: add support for pinning lora adapters in the lru cache
|
hace 5 meses |
|
AlpinDale
|
af43576da0
feat: add MLPSpeculator speculative decoding support (#572)
|
hace 5 meses |
|
AlpinDale
|
34b41e0a87
chore: add coordinator to reduce code duplication in tp and pp
|
hace 6 meses |
|
AlpinDale
|
d0cca80b8b
feat: support sharded tensorizer models
|
hace 6 meses |
|
AlpinDale
|
4d1e613804
chore: minor simplifications
|
hace 6 meses |
|
AlpinDale
|
6cecbbff6a
fix: reduce memory footprint of cuda graph by adding output buffer
|
hace 6 meses |
|
AlpinDale
|
c975bba905
fix: sharded state loader with lora
|
hace 6 meses |
|
AlpinDale
|
e321d80e4e
fix: `prompt_logprobs==0` case
|
hace 6 meses |
|
AlpinDale
|
8d77c69cbd
feat: support image processor and add llava example
|
hace 6 meses |
|
AlpinDale
|
08f639b8aa
remove duplicate seq_lens_tensor
|
hace 6 meses |
|
AlpinDale
|
f40b809d3b
allow using v2 block manager with sliding window
|
hace 6 meses |
|
AlpinDale
|
5b0c11d190
support pipeline parallel pynccl groups
|
hace 6 meses |
|
AlpinDale
|
de62ceb18c
refactor: eliminate parallel worker per-step task scheduling overhead
|
hace 6 meses |
