 AlpinDale
|
3f92035bf1
fix: add `ignored_seq_groups` in `_schedule_chunked_prefill`
|
7 mēneši atpakaļ |
|
AlpinDale
|
17eb1b7eb9
chore: remove ray health check
|
7 mēneši atpakaļ |
|
AlpinDale
|
40bc98b363
chore: use cutlass kernels for fp8 if supported
|
7 mēneši atpakaļ |
|
AlpinDale
|
1d7f5c45b0
feat: add stream_options for chat completions
|
7 mēneši atpakaļ |
|
AlpinDale
|
c975bba905
fix: sharded state loader with lora
|
7 mēneši atpakaļ |
|
AlpinDale
|
b9a5a0ae79
fix: avoid copying prompt/output tokens if penalties arent used
|
7 mēneši atpakaļ |
|
AlpinDale
|
e989ddf2a1
chore: allow using multiple lora modules in llm.generate()
|
7 mēneši atpakaļ |
|
AlpinDale
|
76d6f49bbb
fix: modelscope downloads
|
7 mēneši atpakaļ |
|
AlpinDale
|
f2e94e2184
chore: minor llava cleanups in preparation for llava-next
|
7 mēneši atpakaļ |
|
AlpinDale
|
fde2cda047
chore: update outlines integration from `FSM` to `Guide`
|
7 mēneši atpakaļ |
|
AlpinDale
|
c7bddcdef1
chore: skip for logits_scale == 1.0
|
7 mēneši atpakaļ |
|
AlpinDale
|
d7ebffe2f0
chore: re-add the graceful engine shutdown
|
7 mēneši atpakaļ |
|
AlpinDale
|
ec5b99d075
fix: use named args
|
7 mēneši atpakaļ |
|
AlpinDale
|
e0886ee929
feat: add `ProposerWorkerBase` abstract class
|
7 mēneši atpakaļ |
|
AlpinDale
|
d00a7517e6
fix: tokenizer delay with using LLM class
|
7 mēneši atpakaļ |
|
AlpinDale
|
39b36efabf
fix: mixtral fp8 ckpt loading
|
7 mēneši atpakaļ |
|
AlpinDale
|
e32f506e17
chore: gpu arch guard for cutlass w8a8 kernels
|
7 mēneši atpakaļ |
|
AlpinDale
|
814c1ddeba
feat: add CustomOp interface for device portability
|
7 mēneši atpakaļ |
|
AlpinDale
|
f91f217bf8
fix: do not skip `prompt_logprobs` when `SamplingParams.detokenize=True`
|
7 mēneši atpakaļ |
|
AlpinDale
|
5b5e6dc359
chore: add batch size 1536 and 3072 to moe benchmark
|
7 mēneši atpakaļ |
|
AlpinDale
|
a7fb48acdf
fix: setuptools version in dockerfile for cpu
|
7 mēneši atpakaļ |
|
AlpinDale
|
05d6e43244
fix: `torch.compile()` with mp executor backend
|
7 mēneši atpakaļ |
|
AlpinDale
|
4bdd2f9892
chore: enhance MoE benchmarking
|
7 mēneši atpakaļ |
|
AlpinDale
|
e321d80e4e
fix: `prompt_logprobs==0` case
|
7 mēneši atpakaļ |
|
AlpinDale
|
141c602c39
feat: OpenAI `tools` support named functions
|
7 mēneši atpakaļ |
|
AlpinDale
|
237fa59aea
feat: support CPU/GPU swapping in BlockManagerV2
|
7 mēneši atpakaļ |
|
AlpinDale
|
ba02fb65c9
fix: pos encodings for CPU
|
7 mēneši atpakaļ |
|
AlpinDale
|
90bafca8e3
fix: cuda graphs with sparseml quants
|
7 mēneši atpakaļ |
|
AlpinDale
|
89ee54dcff
update dockerfile and enhance serving benchmark
|
7 mēneši atpakaļ |
|
AlpinDale
|
75f97bc25d
bump flash-attn to remove unnecessary copies in the backend
|
7 mēneši atpakaļ |
