 AlpinDale
|
cac8163f77
fix: request abort crashing pipeline parallel
|
hace 5 meses |
|
AlpinDale
|
45a004874c
chore: allow specifying custom Executor
|
hace 5 meses |
 AlpinDale
|
a26f784240
chore: use the LoRA tokenizer in OpenAI API (#599)
|
hace 5 meses |
|
AlpinDale
|
c0c2b1ac20
fix: get_and_reset only when scheduler outputs are not empty
|
hace 5 meses |
|
AlpinDale
|
b9268be8e8
fix: engine timeout due to request abort
|
hace 5 meses |
|
AlpinDale
|
99680b2d23
feat: soft prompts (#589)
|
hace 5 meses |
|
AlpinDale
|
5240c0da23
fix: avoid unnecessary ray import warnings
|
hace 5 meses |
|
AlpinDale
|
5be90c3859
Mamba infrastrucuture support (#586)
|
hace 5 meses |
|
AlpinDale
|
ae04f57ec1
feat: Pipeline Parallel support (#581)
|
hace 5 meses |
|
AlpinDale
|
0886c361f4
feat: OpenVINO CPU backend (#576)
|
hace 6 meses |
|
AlpinDale
|
c0c336aaa3
refactor: registry for processing model inputs; quick_gelu; clip model support
|
hace 6 meses |
|
AlpinDale
|
4ed1bb9958
chore: add fault tolerance for RayTokenizerGroupPool
|
hace 6 meses |
|
AlpinDale
|
3c7444c89b
fix: asyncio.run hangs in python < 3.12
|
hace 6 meses |
|
AlpinDale
|
6a57861fca
feat: initial XPU support via intel_extension_for_pytorch (#571)
|
hace 6 meses |
|
AlpinDale
|
c482c09a3a
fix: remove duplicated input processing in async engine
|
hace 6 meses |
|
AlpinDale
|
fe21123a1c
feat: TPU support (#570)
|
hace 6 meses |
|
AlpinDale
|
d7ebffe2f0
chore: re-add the graceful engine shutdown
|
hace 6 meses |
|
AlpinDale
|
90ceab32ff
refactor: consolidate prompt args to LLM engines
|
hace 6 meses |
|
AlpinDale
|
de62ceb18c
refactor: eliminate parallel worker per-step task scheduling overhead
|
hace 6 meses |
|
AlpinDale
|
c6a501f682
add multiprocessing executor; make ray optional
|
hace 6 meses |
|
AlpinDale
|
50b7c13db0
refactor: attention selector (#552)
|
hace 6 meses |
|
AlpinDale
|
be8154a8a0
feat: proper embeddings API with e5-mistral-7b support
|
hace 6 meses |
|
AlpinDale
|
3705050cd0
fix python 3.8 syntax
|
hace 6 meses |
|
AlpinDale
|
ef733aee43
implement ExecuteModelData to reduce executor complexity
|
hace 6 meses |
|
AlpinDale
|
29c1b58255
minor logging fixes
|
hace 6 meses |
|
AlpinDale
|
c5fc4a4996
failsafe for later
|
hace 6 meses |
|
AlpinDale
|
aed64884c6
feat: prompt logprobs with chunked prefill (#539)
|
hace 6 meses |
|
AlpinDale
|
199e776722
chore: move ray utils to executor dir
|
hace 7 meses |
|
AlpinDale
|
fca911ee0a
vLLM Upstream Sync (#526)
|
hace 7 meses |
|
AlpinDale
|
9d81716bfd
[v0.5.3] Release Candidate (#388)
|
hace 9 meses |
