 AlpinDale
|
4d4e767838
ci: take one of fixing lint issues
|
vor 4 Monaten |
 AlpinDale
|
0e6c400b13
feat: re-add GGUF (#600)
|
vor 4 Monaten |
|
AlpinDale
|
c9310eeb02
fix: skip loading lm_head for tie_word_embeddings models
|
vor 4 Monaten |
|
AlpinDale
|
1441326ac8
fix: cleanup minicpm-v and port na_vit model
|
vor 4 Monaten |
|
AlpinDale
|
e348ca3540
feat: add support for MiniCPM-V
|
vor 4 Monaten |
|
AlpinDale
|
cb44c8daa8
feat: support FP8 KV Cache scales from compressed-tensors
|
vor 4 Monaten |
|
AlpinDale
|
00503b9fc1
feat: non-uniform quantization via `compressed-tensors` for llama
|
vor 4 Monaten |
|
AlpinDale
|
0429cb2229
fix: only create embeddings and lm_head when necessary for PP
|
vor 4 Monaten |
|
AlpinDale
|
5289c14b24
feat: Asymmetric Tensor Parallel (#594)
|
vor 4 Monaten |
|
AlpinDale
|
9d7beaa5b9
chore: separate kv_scale into k_scale and v_scale
|
vor 4 Monaten |
|
AlpinDale
|
497bf64942
chore: simplify pipeline parallel code in llama
|
vor 4 Monaten |
|
AlpinDale
|
0f4a9ee77b
quantized lm_head (#582)
|
vor 4 Monaten |
|
AlpinDale
|
ae04f57ec1
feat: Pipeline Parallel support (#581)
|
vor 4 Monaten |
|
AlpinDale
|
c5d8028668
fix: no need to redefine supports_vision and supports_lora in model class
|
vor 5 Monaten |
|
AlpinDale
|
56e0b8223c
chore: add base class for LoRA-supported models
|
vor 5 Monaten |
|
AlpinDale
|
690110a051
feat: bitsandbytes quantization
|
vor 5 Monaten |
|
AlpinDale
|
ac79d115b3
add guards for prefix caching, fp8, chunked, etc
|
vor 5 Monaten |
|
AlpinDale
|
f4ea11b982
feat: initial support for activation quantization
|
vor 5 Monaten |
|
AlpinDale
|
c1ed789835
fix: typo in llama.py
|
vor 5 Monaten |
|
AlpinDale
|
656459fd84
make fp8_e4m3 work on nvidia
|
vor 5 Monaten |
|
AlpinDale
|
9e73559eba
make use of batched rotary embedding kernels to support long context lora
|
vor 5 Monaten |
|
AlpinDale
|
2ecfa98da9
re-fix mistral nemo
|
vor 5 Monaten |
|
AlpinDale
|
50b7c13db0
refactor: attention selector (#552)
|
vor 5 Monaten |
|
AlpinDale
|
54a4cef647
add bias and tie word embedding support for llama
|
vor 5 Monaten |
|
AlpinDale
|
639e48e47d
fix: mistral nemo
|
vor 5 Monaten |
|
AlpinDale
|
b178ae4b4a
chore: generalize linear_method to be quant_method (#540)
|
vor 5 Monaten |
|
AlpinDale
|
e7b1368156
feat: Phi3 support
|
vor 6 Monaten |
|
AlpinDale
|
fca911ee0a
vLLM Upstream Sync (#526)
|
vor 6 Monaten |
|
AlpinDale
|
9d81716bfd
[v0.5.3] Release Candidate (#388)
|
vor 8 Monaten |
|
AlpinDale
|
f8dfac6372
chore: attention refactor and upstream sync apr01 (#365)
|
vor 9 Monaten |
