 AlpinDale
|
2dfa4e47e6
chore: set seed for dummy weights init
|
vor 5 Monaten |
|
AlpinDale
|
9d7beaa5b9
chore: separate kv_scale into k_scale and v_scale
|
vor 5 Monaten |
|
AlpinDale
|
b82c39772c
chore: allow quantizing all layers of deepseek-v2
|
vor 5 Monaten |
|
AlpinDale
|
517676249c
chore: update the compressed-tensors config
|
vor 6 Monaten |
|
AlpinDale
|
690110a051
feat: bitsandbytes quantization
|
vor 6 Monaten |
|
AlpinDale
|
7d0884de9a
fix mistral v0.3 weight loading
|
vor 6 Monaten |
|
AlpinDale
|
f4ea11b982
feat: initial support for activation quantization
|
vor 6 Monaten |
|
AlpinDale
|
94ba676ee0
fix: torch.uniform_ doesn't support FP8, fix for dummy weights
|
vor 6 Monaten |
|
AlpinDale
|
3bbfd65549
feat: support hub model ID when offline
|
vor 6 Monaten |
|
AlpinDale
|
46159b107a
formatting: pt1
|
vor 7 Monaten |
 AlpinDale
|
fca911ee0a
vLLM Upstream Sync (#526)
|
vor 7 Monaten |
