 AlpinDale
|
9a7d5514c4
feat: introduce MQAphroditeEngine (#1056)
|
hai 3 semanas |
|
AlpinDale
|
638c08d9dc
fix: clean shutdown issues (#1047)
|
hai 4 semanas |
|
AlpinDale
|
05be6085ec
core: factor out input preprocessing into a separate class (#1039)
|
hai 4 semanas |
|
AlpinDale
|
ddaefd8d38
chore: remove engine_use_ray (#1024)
|
hai 4 semanas |
|
AlpinDale
|
f561a54a43
core: fix async postprocessor in case of preemption (#1000)
|
hai 1 mes |
|
AlpinDale
|
09dab16f82
core: improve async postproc + multi-step performance (#983)
|
hai 1 mes |
|
AlpinDale
|
0dfa6b60ec
core: support logprobs with multi-step scheduling (#963)
|
hai 1 mes |
|
AlpinDale
|
5be6225f38
core: support multi-step scheduling w/ async post-processor (#955)
|
hai 1 mes |
|
AlpinDale
|
0e2bfccda0
core: add virtual engine for async outproc (#939)
|
hai 1 mes |
|
AlpinDale
|
15cb8d5c26
xpu: support pipeline parallel (#932)
|
hai 1 mes |
|
AlpinDale
|
f7f3fed265
feat: add async postprocessor (#925)
|
hai 1 mes |
|
AlpinDale
|
b1492c1529
core: add multi-step scheduling support for the synchronous engine (#914)
|
hai 1 mes |
|
AlpinDale
|
22a4cd4595
core: fix spec decode metrics and envs circular import (#889)
|
hai 1 mes |
|
AlpinDale
|
901900854e
chore: consolidate environment variables within one file (#882)
|
hai 1 mes |
|
AlpinDale
|
b5aa11020b
api: fix crashes under very high loads (#878)
|
hai 1 mes |
|
AlpinDale
|
5bd4473bb6
async: avoid premature exit in the async generator (#872)
|
hai 1 mes |
|
AlpinDale
|
48a8693aed
feat: multi-step scheduling (#831)
|
hai 2 meses |
 AlpinDale
|
dcb794a340
fix: revert incorrect commit
|
hai 3 meses |
|
AlpinDale
|
76367b5ae7
wip
|
hai 3 meses |
|
AlpinDale
|
9d9722b1c1
fix: metrics endpoint with RPC server (#747)
|
hai 4 meses |
|
AlpinDale
|
7debd35ca2
fix: shut down ray dag workers cleanly (#692)
|
hai 4 meses |
|
AlpinDale
|
19ad952dd4
chore: better stream termination in async engine (#672)
|
hai 4 meses |
|
AlpinDale
|
62111fab17
feat: allow serving encoder-decoder models in the API server (#664)
|
hai 4 meses |
|
AlpinDale
|
ed9a6f97c1
fix: kill api server when pinging dead engine (#660)
|
hai 4 meses |
|
AlpinDale
|
77c4fbd5c9
fix: better async request cancellation (#641)
|
hai 4 meses |
|
AlpinDale
|
f1d0b77c92
[0.6.0] Release Candidate (#481)
|
hai 4 meses |
|
AlpinDale
|
9d81716bfd
[v0.5.3] Release Candidate (#388)
|
hai 8 meses |
|
AlpinDale
|
0f6d56b07f
feat: model executor refactor (#367)
|
hai 9 meses |
|
AlpinDale
|
b361096463
fix: tokenizer when using ray (#366)
|
hai 9 meses |
|
AlpinDale
|
f8dfac6372
chore: attention refactor and upstream sync apr01 (#365)
|
hai 9 meses |
