 AlpinDale
|
2b85ffb1a5
chore: minor cleanups
|
hai 4 meses |
|
AlpinDale
|
4d4e767838
ci: take one of fixing lint issues
|
hai 4 meses |
 AlpinDale
|
0e6c400b13
feat: re-add GGUF (#600)
|
hai 4 meses |
|
AlpinDale
|
edffcecc67
chore: add proper logging for spec decoding verification
|
hai 4 meses |
|
AlpinDale
|
d357341203
chore: add pipeline parallel support for Qwen
|
hai 4 meses |
|
AlpinDale
|
98f9dbd734
feat: Triton Kernels for Punica (#613)
|
hai 4 meses |
|
AlpinDale
|
07cc8a56bb
fix: add nemotron to PP_SUPPORTED_MODELS
|
hai 5 meses |
|
AlpinDale
|
ea838abb6b
fix: disable enforce_eager for bnb
|
hai 5 meses |
|
AlpinDale
|
fce2c2e304
fix: support ignore patterns in model loader
|
hai 5 meses |
|
AlpinDale
|
cb44c8daa8
feat: support FP8 KV Cache scales from compressed-tensors
|
hai 5 meses |
|
AlpinDale
|
ba371fbbbd
feat: AWQ marlin kernels (#603)
|
hai 5 meses |
|
AlpinDale
|
a4cbcfe59f
feat: disable logprob serialization to CPU for spec decode
|
hai 5 meses |
|
AlpinDale
|
9be43994fe
feat: fbgemm quantization support (#601)
|
hai 5 meses |
|
AlpinDale
|
45a004874c
chore: allow specifying custom Executor
|
hai 5 meses |
|
AlpinDale
|
b7a2d52e47
fix: allow using mp executor for pipeline parallel
|
hai 5 meses |
|
AlpinDale
|
6671e3a162
feat: add CPU offloading support (#598)
|
hai 5 meses |
|
AlpinDale
|
ee2c5d34da
feat: add fp8 channel-wise weight quantization support
|
hai 5 meses |
|
AlpinDale
|
6c4c20652b
feat: pipeline parallel support for mixtral
|
hai 5 meses |
|
AlpinDale
|
5289c14b24
feat: Asymmetric Tensor Parallel (#594)
|
hai 5 meses |
|
AlpinDale
|
ddb28a80a3
fix: bump torch for rocm, unify CUDA_VISIBLE_DEVICES for cuda and rocm
|
hai 5 meses |
|
AlpinDale
|
99680b2d23
feat: soft prompts (#589)
|
hai 5 meses |
|
AlpinDale
|
5761ef8c35
feat: gemma-2 support
|
hai 5 meses |
|
AlpinDale
|
1ff6d4c3d7
feat: support pipeline parallel on indivisible GPU count (#587)
|
hai 5 meses |
|
AlpinDale
|
4f7d212b70
feat: remove vision language config
|
hai 5 meses |
|
AlpinDale
|
bdf1cc1aec
fix: allow using custom all reduce when pp_size > 1
|
hai 5 meses |
|
AlpinDale
|
5240c0da23
fix: avoid unnecessary ray import warnings
|
hai 5 meses |
|
AlpinDale
|
5be90c3859
Mamba infrastrucuture support (#586)
|
hai 5 meses |
|
AlpinDale
|
ae04f57ec1
feat: Pipeline Parallel support (#581)
|
hai 5 meses |
|
AlpinDale
|
dd378ea063
feat: MLPSpeculator with tensor parallel
|
hai 5 meses |
|
AlpinDale
|
3a0fdf7b9b
chore: remove `image_input_type` from VLM config
|
hai 5 meses |
