 AlpinDale
|
cc13f2d631
hack: get_last_latency crash
|
il y a 2 semaines |
 AlpinDale
|
a3c03db735
fix: inline model loading conflicts with lora (#930)
|
il y a 2 semaines |
|
AlpinDale
|
59d1d59028
api: support aphrodite_config.yaml with inline loading (#929)
|
il y a 2 semaines |
|
AlpinDale
|
d46e70ac98
api: add inline model loading (#928)
|
il y a 2 semaines |
|
AlpinDale
|
8d9f1fd4e6
feat: add single user mode (#927)
|
il y a 2 semaines |
|
AlpinDale
|
53d0ba7c7c
api: add endpoint for loading and unloading the model (#926)
|
il y a 2 semaines |
|
AlpinDale
|
f7f3fed265
feat: add async postprocessor (#925)
|
il y a 2 semaines |
|
AlpinDale
|
5cb2e998d8
quants: update compressed tensors lifecycle to remove `prefix` from `create_weights` (#924)
|
il y a 3 semaines |
|
AlpinDale
|
0c6d90dade
neuron: add support for tensor parallelism (#923)
|
il y a 3 semaines |
|
AlpinDale
|
2940da2c7b
distributed: fix custom allreduce p2p cache file generation (#922)
|
il y a 3 semaines |
|
AlpinDale
|
5d9021969c
quants: update `qqq` and `gptq_marlin_24` to use AphroditeParameters (#921)
|
il y a 3 semaines |
|
AlpinDale
|
9c9b2dd843
core: improve warmup times for prefix caching in block manager v2 (#920)
|
il y a 3 semaines |
|
AlpinDale
|
0c162c8dad
api: use fp32 for base64 embeddings (#919)
|
il y a 3 semaines |
|
AlpinDale
|
3b684a8a54
spec decode: streamline batch expansion tensor manipulation (#918)
|
il y a 3 semaines |
|
AlpinDale
|
fce970a846
feat: multi-image input support for Phi3V (#917)
|
il y a 3 semaines |
|
AlpinDale
|
178c2141d4
fix: phi3v crash with unusual image sizes (#916)
|
il y a 3 semaines |
|
AlpinDale
|
f61acdd3ec
api: add json_schema to OpenAI server (#915)
|
il y a 3 semaines |
|
AlpinDale
|
b1492c1529
core: add multi-step scheduling support for the synchronous engine (#914)
|
il y a 3 semaines |
|
AlpinDale
|
799667737b
quantization: update marlin to use `AphroditeParameters` (#913)
|
il y a 3 semaines |
|
AlpinDale
|
16e5b2be8b
fix: empty prompt crashing the server (#912)
|
il y a 3 semaines |
|
AlpinDale
|
673621a3d2
xpu: refactor the model runner for tensor parallelism (#910)
|
il y a 3 semaines |
|
AlpinDale
|
d69273bd2b
ray: better error when placement group topology is incorrect (#906)
|
il y a 3 semaines |
|
AlpinDale
|
6fbab320e7
api: error suppression cleanup + timeout suppression on aborts (#905)
|
il y a 3 semaines |
|
AlpinDale
|
ab533e0e60
spec decode: fix logprobs when using speculative decoding (#904)
|
il y a 3 semaines |
|
AlpinDale
|
afc9a28aa0
chore: add AphroditeParameter support for FP8 quant (#902)
|
il y a 3 semaines |
|
AlpinDale
|
2a60b8f8c9
kernel: do not compile machete for cuda 11 and below (#901)
|
il y a 3 semaines |
|
AlpinDale
|
64c05b969a
fix: `ShardedStateLoader` with fp8 quant (#900)
|
il y a 3 semaines |
|
AlpinDale
|
132aa2abe4
spec decode: add support for EAGLE (#899)
|
il y a 3 semaines |
|
AlpinDale
|
bfc3da41ae
feat: add torch.compile for GemmaRMSNorm (#898)
|
il y a 3 semaines |
|
AlpinDale
|
a00ab49e21
api: add client timeouts for the ZeroMQ server (#897)
|
il y a 3 semaines |
