 AlpinDale
|
ac79d115b3
add guards for prefix caching, fp8, chunked, etc
|
vor 6 Monaten |
|
AlpinDale
|
f4ea11b982
feat: initial support for activation quantization
|
vor 6 Monaten |
|
AlpinDale
|
c1ed789835
fix: typo in llama.py
|
vor 6 Monaten |
|
AlpinDale
|
656459fd84
make fp8_e4m3 work on nvidia
|
vor 6 Monaten |
|
AlpinDale
|
9e73559eba
make use of batched rotary embedding kernels to support long context lora
|
vor 6 Monaten |
|
AlpinDale
|
2ecfa98da9
re-fix mistral nemo
|
vor 6 Monaten |
 AlpinDale
|
50b7c13db0
refactor: attention selector (#552)
|
vor 6 Monaten |
|
AlpinDale
|
54a4cef647
add bias and tie word embedding support for llama
|
vor 6 Monaten |
|
AlpinDale
|
639e48e47d
fix: mistral nemo
|
vor 6 Monaten |
|
AlpinDale
|
b178ae4b4a
chore: generalize linear_method to be quant_method (#540)
|
vor 6 Monaten |
|
AlpinDale
|
e7b1368156
feat: Phi3 support
|
vor 7 Monaten |
|
AlpinDale
|
fca911ee0a
vLLM Upstream Sync (#526)
|
vor 7 Monaten |
|
AlpinDale
|
9d81716bfd
[v0.5.3] Release Candidate (#388)
|
vor 9 Monaten |
|
AlpinDale
|
f8dfac6372
chore: attention refactor and upstream sync apr01 (#365)
|
vor 10 Monaten |
 sgsdxzy
|
6ebac34dc1
chore: cleaner pre-llamafied Yi implementation (#352)
|
vor 10 Monaten |
|
AlpinDale
|
681e94611f
fix: restore backwards compatibility with old Yi models (#351)
|
vor 10 Monaten |
|
AlpinDale
|
da223153c6
feat&fix: cohere support and missing GPU blocks (#333)
|
vor 10 Monaten |
|
AlpinDale
|
e42a78381a
feat: switch from pylint to ruff (#322)
|
vor 11 Monaten |
|
AlpinDale
|
9810daa699
feat: INT8 KV Cache (#298)
|
vor 11 Monaten |
|
AlpinDale
|
e31c6f0b45
feat: refactor modeling logic and support more models (#274)
|
vor 11 Monaten |
|
AlpinDale
|
842912d022
feat: on-the-fly gguf conversion (#250)
|
vor 1 Jahr |
|
AlpinDale
|
c3a221eb02
feat: GGUF, QuIP#, and Marlin support (#228)
|
vor 1 Jahr |
|
AlpinDale
|
c0aac15421
feat: S-LoRA support (#222)
|
vor 1 Jahr |
|
AlpinDale
|
8fa608aeb7
feat: replace Ray with NCCL for control plane comms (#221)
|
vor 1 Jahr |
|
AlpinDale
|
f013d714c0
chore: merge dev branch into main (#177)
|
vor 1 Jahr |
|
AlpinDale
|
2755a48d51
merge dev branch into main (#153)
|
vor 1 Jahr |
|
AlpinDale
|
887e03669a
feat: add exllamav2 for GPTQ (#99)
|
vor 1 Jahr |
|
AlpinDale
|
74604eb252
fix: pylint complaints (#91)
|
vor 1 Jahr |
|
AlpinDale
|
efc6f7fbec
chore: reformats (#90)
|
vor 1 Jahr |
|
AlpinDale
|
a6a4220fa6
feat: refactor megatron and quants (#57)
|
vor 1 Jahr |
