 AlpinDale
|
9bbc75d2e3
wip
|
vor 7 Monaten |
|
AlpinDale
|
60af35bc34
wip
|
vor 7 Monaten |
|
AlpinDale
|
74cb1aad4e
wip
|
vor 7 Monaten |
|
AlpinDale
|
5d965b34a7
bitnet -> bitblas in reqs
|
vor 7 Monaten |
|
AlpinDale
|
5884e0b904
add bitnetforcausallm support
|
vor 7 Monaten |
|
AlpinDale
|
2649f3f14e
aqlm works on pascal
|
vor 7 Monaten |
|
AlpinDale
|
ac79d115b3
add guards for prefix caching, fp8, chunked, etc
|
vor 7 Monaten |
|
AlpinDale
|
344ddaac5a
properly disable speculative decoding
|
vor 7 Monaten |
|
AlpinDale
|
696f2cd59c
add phi3_small support with blocksparse attention
|
vor 7 Monaten |
|
AlpinDale
|
0d15aa3ab3
fix prefix caching for block manager v2
|
vor 7 Monaten |
|
AlpinDale
|
7d0884de9a
fix mistral v0.3 weight loading
|
vor 7 Monaten |
|
AlpinDale
|
e8b7f53321
allow prompt token IDs in the logits processor api
|
vor 7 Monaten |
|
AlpinDale
|
f4ea11b982
feat: initial support for activation quantization
|
vor 7 Monaten |
 Drake
|
e1a142c179
Fix OpenAI chat completions compatibility (#559)
|
vor 7 Monaten |
|
AlpinDale
|
5b0c11d190
support pipeline parallel pynccl groups
|
vor 7 Monaten |
|
AlpinDale
|
f6250c5516
move dockerfiles to root; fix cpu build
|
vor 7 Monaten |
|
AlpinDale
|
d8667fcb98
improve gptq_marlin_24 prefill performance
|
vor 7 Monaten |
|
AlpinDale
|
eb2c5c77df
feat: enforce the max possible seqlen
|
vor 7 Monaten |
|
AlpinDale
|
19a959a03e
prioritize user selection for attention
|
vor 7 Monaten |
|
AlpinDale
|
c1ed789835
fix: typo in llama.py
|
vor 7 Monaten |
|
AlpinDale
|
4e1ae004da
make mp the default distributed backend
|
vor 7 Monaten |
|
AlpinDale
|
de62ceb18c
refactor: eliminate parallel worker per-step task scheduling overhead
|
vor 7 Monaten |
|
AlpinDale
|
656459fd84
make fp8_e4m3 work on nvidia
|
vor 7 Monaten |
|
AlpinDale
|
6e626b902c
fix cutlass w8a8 kernels for cuda stream
|
vor 7 Monaten |
 AlpinDale
|
3bdeb3e116
fix: clang formatting for all kernels (#558)
|
vor 7 Monaten |
|
AlpinDale
|
04d22bf1a9
add clang-format
|
vor 7 Monaten |
|
AlpinDale
|
60e74e92fd
add rope_scaling arg
|
vor 7 Monaten |
|
AlpinDale
|
b8b63eb5ca
fix head_size check for flash attention backend
|
vor 7 Monaten |
|
AlpinDale
|
8077af0b2f
add lora support for phi
|
vor 7 Monaten |
|
AlpinDale
|
295cfb2f39
add rope scaling for qwen2
|
vor 7 Monaten |
