 AlpinDale
|
2b5af25923
add perplexity example
|
vor 10 Monaten |
 drummerv
|
e59dd4a90d
fix: openai gguf chat template (#312)
|
vor 10 Monaten |
|
AlpinDale
|
b3df2351c8
readme: update with bsz1 graph
|
vor 10 Monaten |
|
AlpinDale
|
434dc19961
CI: fix build failure for cuda versions with no torch wheels
|
vor 10 Monaten |
 AlpinDale
|
968bde81bf
fix: tensor parallel with GPTQ and AWQ quants (#307)
|
vor 10 Monaten |
|
AlpinDale
|
ff898c2c80
bump version to 0.5.0 (#303)
|
vor 10 Monaten |
|
AlpinDale
|
c41462cfcd
feat: exllamav2 quantization (#305)
|
vor 10 Monaten |
|
AlpinDale
|
3a045ebfde
fix: escape tags in loguru (#304)
|
vor 10 Monaten |
|
AlpinDale
|
9ec611090d
chore: build for more cuda versions
|
vor 10 Monaten |
|
AlpinDale
|
c2d77b1822
chore: logging refactor (#302)
|
vor 10 Monaten |
|
AlpinDale
|
132d9927cb
fix: speedup runtime update script
|
vor 10 Monaten |
 Stefan Gligorijevic
|
7380c2c3ff
chore: update gxx to 11.3 (#282)
|
vor 10 Monaten |
 Aykut Akgün
|
cbe37e8b18
fix: speed up cuda home detection (#288)
|
vor 10 Monaten |
|
AlpinDale
|
a98babfb74
fix: bnb on Turing GPUs (#299)
|
vor 10 Monaten |
|
AlpinDale
|
49793d7c5a
fix: bump bnb kernels to sm_80 due to async stream copies
|
vor 10 Monaten |
|
AlpinDale
|
9810daa699
feat: INT8 KV Cache (#298)
|
vor 10 Monaten |
|
AlpinDale
|
82955ba440
fix: backport bnb kernels (#297)
|
vor 10 Monaten |
 Pyroserenus
|
951077de65
chore: update klite.embd with current version (#296)
|
vor 10 Monaten |
 sgsdxzy
|
94c1543cae
fix: typo in marlin kernel path (#295)
|
vor 10 Monaten |
|
AlpinDale
|
e0c35bb353
feat: bitsandbytes and `--load-in{4,8}bit` support (#294)
|
vor 10 Monaten |
|
AlpinDale
|
705821a7fe
feat: AQLM quantization support (#293)
|
vor 10 Monaten |
|
AlpinDale
|
a1d8ab9f3e
fix: lora on quantized models (barred gguf) (#292)
|
vor 10 Monaten |
|
AlpinDale
|
2d3d44b3e9
chore: add health check for ray workers (#290)
|
vor 10 Monaten |
|
AlpinDale
|
ac82b67f75
feat: naive context shift and various QoL changes (#289)
|
vor 10 Monaten |
|
AlpinDale
|
f35d15e632
fix: arg detection for kobold api launch (#286)
|
vor 10 Monaten |
|
AlpinDale
|
72229a94da
feat: better marlin kernels (#285)
|
vor 10 Monaten |
|
AlpinDale
|
769b069e2e
AttributeError fix in OpenAI server
|
vor 10 Monaten |
|
AlpinDale
|
23a7fd8cda
remove ooba endpoint, fix and add deprecation warning for kobold endpoint, fix case where kobold endpoint was always launched with openai (#284)
|
vor 10 Monaten |
|
AlpinDale
|
13d850334e
fix: navi support (#283)
|
vor 10 Monaten |
|
AlpinDale
|
9fa99215f8
feat: add cubic sampling (#280)
|
vor 10 Monaten |
