 AlpinDale
|
a94de94c44
refactor: combine the prefill and decode into a single API (#553)
|
hai 6 meses |
 AlpinDale
|
342346afda
improve hashing function
|
hai 6 meses |
|
AlpinDale
|
be8154a8a0
feat: proper embeddings API with e5-mistral-7b support
|
hai 6 meses |
|
AlpinDale
|
197a6d2c16
auto disable speculative decoding by the running queue size
|
hai 6 meses |
|
AlpinDale
|
8b56dc4347
dict -> torch.Tensor for blocks_to_swap
|
hai 6 meses |
|
AlpinDale
|
21ce19b3ea
blocks_to_copy dict -> torch.Tensor
|
hai 6 meses |
|
AlpinDale
|
ef733aee43
implement ExecuteModelData to reduce executor complexity
|
hai 6 meses |
|
AlpinDale
|
79901b76de
logprobs for target model (spec decoding)
|
hai 6 meses |
|
AlpinDale
|
2351a0e2cd
feat: FlashInfer backend for decoding phase (#548)
|
hai 6 meses |
|
AlpinDale
|
b1555eb208
add new grafana metrics
|
hai 6 meses |
|
AlpinDale
|
aed64884c6
feat: prompt logprobs with chunked prefill (#539)
|
hai 6 meses |
|
AlpinDale
|
9d81716bfd
[v0.5.3] Release Candidate (#388)
|
hai 8 meses |
|
AlpinDale
|
9181fa0396
feat: Triton kernels for sampling (#383)
|
hai 9 meses |
|
AlpinDale
|
f8dfac6372
chore: attention refactor and upstream sync apr01 (#365)
|
hai 9 meses |
|
AlpinDale
|
e42a78381a
feat: switch from pylint to ruff (#322)
|
hai 10 meses |
|
AlpinDale
|
9810daa699
feat: INT8 KV Cache (#298)
|
hai 10 meses |
|
AlpinDale
|
ac82b67f75
feat: naive context shift and various QoL changes (#289)
|
hai 10 meses |
|
AlpinDale
|
657aec0cbd
refactor: OpenAI endpoint (#261)
|
hai 10 meses |
|
AlpinDale
|
4d04ade9ef
feat: fine-grained seeds (#279)
|
hai 11 meses |
|
AlpinDale
|
d2db4143fa
feat: add grafana for metrics (#240)
|
hai 11 meses |
|
AlpinDale
|
c0aac15421
feat: S-LoRA support (#222)
|
hai 1 ano |
|
AlpinDale
|
8fa608aeb7
feat: replace Ray with NCCL for control plane comms (#221)
|
hai 1 ano |
|
AlpinDale
|
f013d714c0
chore: merge dev branch into main (#177)
|
hai 1 ano |
|
AlpinDale
|
2755a48d51
merge dev branch into main (#153)
|
hai 1 ano |
 50h100a
|
fa0ae5a2c9
feat: new mirostatv2 implementation (#96)
|
hai 1 ano |
|
AlpinDale
|
efc6f7fbec
chore: reformats (#90)
|
hai 1 ano |
|
AlpinDale
|
e6be0118c9
feat: prompt logprobs and batched samplers (#77)
|
hai 1 ano |
|
AlpinDale
|
75c27d3e65
massive overhaul
|
hai 1 ano |
|
AlpinDale
|
6dfca14e1f
compute logprobs with log_softmax instead of log
|
hai 1 ano |
|
AlpinDale
|
6b9561ef07
adapt TGI incremental detokenization
|
hai 1 ano |
