 AlpinDale
|
2dfa4e47e6
chore: set seed for dummy weights init
|
hace 5 meses |
|
AlpinDale
|
9d7beaa5b9
chore: separate kv_scale into k_scale and v_scale
|
hace 5 meses |
|
AlpinDale
|
b82c39772c
chore: allow quantizing all layers of deepseek-v2
|
hace 5 meses |
|
AlpinDale
|
517676249c
chore: update the compressed-tensors config
|
hace 6 meses |
|
AlpinDale
|
690110a051
feat: bitsandbytes quantization
|
hace 6 meses |
|
AlpinDale
|
7d0884de9a
fix mistral v0.3 weight loading
|
hace 6 meses |
|
AlpinDale
|
f4ea11b982
feat: initial support for activation quantization
|
hace 6 meses |
|
AlpinDale
|
94ba676ee0
fix: torch.uniform_ doesn't support FP8, fix for dummy weights
|
hace 6 meses |
|
AlpinDale
|
3bbfd65549
feat: support hub model ID when offline
|
hace 6 meses |
|
AlpinDale
|
46159b107a
formatting: pt1
|
hace 7 meses |
 AlpinDale
|
fca911ee0a
vLLM Upstream Sync (#526)
|
hace 7 meses |
