| .. |
|
attention
|
6fc1ec6e9a
fix redirects and improve low level debugging
|
7 mesiacov pred |
|
common
|
723c6acb84
re-add ngram speculative decoding
|
7 mesiacov pred |
|
distributed
|
fd1ce09581
allow multiple tp groups
|
7 mesiacov pred |
|
endpoints
|
7e3887f8f8
use random seed if -1
|
7 mesiacov pred |
|
engine
|
29c1b58255
minor logging fixes
|
7 mesiacov pred |
|
executor
|
fd1ce09581
allow multiple tp groups
|
7 mesiacov pred |
|
kv_quant
|
e42a78381a
feat: switch from pylint to ruff (#322)
|
1 rok pred |
|
lora
|
e87c32bed3
feat: full tensor parallel for LoRA layers (#545)
|
7 mesiacov pred |
|
modeling
|
0751a2ecf6
fix expert_ids shape in Moe
|
7 mesiacov pred |
|
processing
|
5529304d1f
fix sampling with n>1
|
7 mesiacov pred |
|
quantization
|
c154578c97
gptq_marlin: 8bit GPTQ support
|
7 mesiacov pred |
|
spec_decode
|
723c6acb84
re-add ngram speculative decoding
|
7 mesiacov pred |
|
task_handler
|
aed64884c6
feat: prompt logprobs with chunked prefill (#539)
|
8 mesiacov pred |
|
transformers_utils
|
12c22ddf98
fix modelscope tokenizer download
|
7 mesiacov pred |
|
__init__.py
|
199e776722
chore: move ray utils to executor dir
|
8 mesiacov pred |
|
py.typed
|
1c988a48b2
fix logging and add py.typed
|
1 rok pred |
AlpinDale
c154578c97
gptq_marlin: 8bit GPTQ support