 AlpinDale
|
86bf2cc4f3
core: rename `PromptInputs,inputs` -> `PromptType,prompt` (#1080)
|
hace 3 semanas |
|
AlpinDale
|
9a7d5514c4
feat: introduce MQAphroditeEngine (#1056)
|
hace 1 mes |
|
AlpinDale
|
638c08d9dc
fix: clean shutdown issues (#1047)
|
hace 1 mes |
|
AlpinDale
|
05be6085ec
core: factor out input preprocessing into a separate class (#1039)
|
hace 1 mes |
|
AlpinDale
|
055c8905a3
api: add sampling/engine option to return only deltas or final output (#1035)
|
hace 1 mes |
|
AlpinDale
|
ddaefd8d38
chore: remove engine_use_ray (#1024)
|
hace 1 mes |
|
AlpinDale
|
f561a54a43
core: fix async postprocessor in case of preemption (#1000)
|
hace 1 mes |
|
AlpinDale
|
55fa2b9c54
fix: raise exception when accessing logger for disable_log_stats=True case (#984)
|
hace 1 mes |
|
AlpinDale
|
09dab16f82
core: improve async postproc + multi-step performance (#983)
|
hace 1 mes |
|
AlpinDale
|
b3f6eeb1d2
vlm: increase the default `max_num_batched_tokens` for multimodal models (#973)
|
hace 1 mes |
|
AlpinDale
|
0dfa6b60ec
core: support logprobs with multi-step scheduling (#963)
|
hace 1 mes |
|
AlpinDale
|
34e8606e81
vlm: do not allow max_model_len overflow (#962)
|
hace 1 mes |
|
AlpinDale
|
5be6225f38
core: support multi-step scheduling w/ async post-processor (#955)
|
hace 1 mes |
|
AlpinDale
|
0e2bfccda0
core: add virtual engine for async outproc (#939)
|
hace 1 mes |
|
AlpinDale
|
15cb8d5c26
xpu: support pipeline parallel (#932)
|
hace 1 mes |
|
AlpinDale
|
8d9f1fd4e6
feat: add single user mode (#927)
|
hace 1 mes |
|
AlpinDale
|
f7f3fed265
feat: add async postprocessor (#925)
|
hace 1 mes |
|
AlpinDale
|
b1492c1529
core: add multi-step scheduling support for the synchronous engine (#914)
|
hace 1 mes |
|
AlpinDale
|
16e5b2be8b
fix: empty prompt crashing the server (#912)
|
hace 1 mes |
|
AlpinDale
|
22a4cd4595
core: fix spec decode metrics and envs circular import (#889)
|
hace 1 mes |
|
AlpinDale
|
901900854e
chore: consolidate environment variables within one file (#882)
|
hace 1 mes |
|
AlpinDale
|
db6a50fd5c
async: disable multi-step scheduling for sync engine (#880)
|
hace 1 mes |
 AlpinDale
|
2242cb25dc
fix: unbound tokenizer error
|
hace 2 meses |
|
AlpinDale
|
3d83e64f8e
feat: add metrics for prefix cache hit rate (#829)
|
hace 2 meses |
|
AlpinDale
|
9d9722b1c1
fix: metrics endpoint with RPC server (#747)
|
hace 4 meses |
|
AlpinDale
|
89a2c6dee1
chore: refactor `MultiModalConfig` initialization and profiling (#745)
|
hace 4 meses |
|
AlpinDale
|
0b8b407b6d
feat: support profiling with multiple multi-modal inputs per prompt (#712)
|
hace 4 meses |
|
AlpinDale
|
f76f2a5af0
feat: add aphrodite plugin system (#705)
|
hace 4 meses |
|
AlpinDale
|
7debd35ca2
fix: shut down ray dag workers cleanly (#692)
|
hace 4 meses |
|
AlpinDale
|
bf88c8567e
feat: mamba model support (#674)
|
hace 4 meses |
