 AlpinDale
|
a4cbcfe59f
feat: disable logprob serialization to CPU for spec decode
|
vor 5 Monaten |
|
AlpinDale
|
45a004874c
chore: allow specifying custom Executor
|
vor 5 Monaten |
 AlpinDale
|
6671e3a162
feat: add CPU offloading support (#598)
|
vor 5 Monaten |
|
AlpinDale
|
cf381a0c54
OpenAI API Refactor (#591)
|
vor 5 Monaten |
|
AlpinDale
|
99680b2d23
feat: soft prompts (#589)
|
vor 5 Monaten |
|
AlpinDale
|
4f7d212b70
feat: remove vision language config
|
vor 5 Monaten |
|
AlpinDale
|
3a0fdf7b9b
chore: remove `image_input_type` from VLM config
|
vor 5 Monaten |
|
AlpinDale
|
7253e9052d
feat: integrate typical acceptance sampling for spec decoding
|
vor 5 Monaten |
|
AlpinDale
|
b8a19ba27f
chore: extend aphrodite metrics logging api
|
vor 5 Monaten |
|
AlpinDale
|
0886c361f4
feat: OpenVINO CPU backend (#576)
|
vor 5 Monaten |
|
AlpinDale
|
c0c336aaa3
refactor: registry for processing model inputs; quick_gelu; clip model support
|
vor 5 Monaten |
|
AlpinDale
|
80ac1cdc8f
fix: add args for the draft tp
|
vor 5 Monaten |
|
AlpinDale
|
d44ac8e497
fix: `--preemption_mode` -> `--preemption-mode`
|
vor 6 Monaten |
|
AlpinDale
|
6a57861fca
feat: initial XPU support via intel_extension_for_pytorch (#571)
|
vor 6 Monaten |
|
AlpinDale
|
fe21123a1c
feat: TPU support (#570)
|
vor 6 Monaten |
|
AlpinDale
|
fa58ba87a3
fix: only set executor backend to mp if not multi-node
|
vor 6 Monaten |
|
AlpinDale
|
bba89fc6d3
chore: make the automatic rope scaling behave properly with rope_scaling arg, add rope theta
|
vor 6 Monaten |
|
AlpinDale
|
c61d9f1aa3
fix: lora_dtype value in args
|
vor 6 Monaten |
|
AlpinDale
|
ec5b99d075
fix: use named args
|
vor 6 Monaten |
|
AlpinDale
|
237fa59aea
feat: support CPU/GPU swapping in BlockManagerV2
|
vor 6 Monaten |
|
AlpinDale
|
8d77c69cbd
feat: support image processor and add llava example
|
vor 6 Monaten |
|
AlpinDale
|
690110a051
feat: bitsandbytes quantization
|
vor 6 Monaten |
|
AlpinDale
|
f40b809d3b
allow using v2 block manager with sliding window
|
vor 6 Monaten |
|
AlpinDale
|
ac79d115b3
add guards for prefix caching, fp8, chunked, etc
|
vor 6 Monaten |
|
AlpinDale
|
f6250c5516
move dockerfiles to root; fix cpu build
|
vor 6 Monaten |
|
AlpinDale
|
4e1ae004da
make mp the default distributed backend
|
vor 6 Monaten |
|
AlpinDale
|
656459fd84
make fp8_e4m3 work on nvidia
|
vor 6 Monaten |
|
AlpinDale
|
60e74e92fd
add rope_scaling arg
|
vor 6 Monaten |
|
AlpinDale
|
9e73559eba
make use of batched rotary embedding kernels to support long context lora
|
vor 6 Monaten |
|
AlpinDale
|
7bcff4ac03
implement sharded state dict
|
vor 6 Monaten |
