 AlpinDale
|
132aa2abe4
spec decode: add support for EAGLE (#899)
|
hai 3 semanas |
|
AlpinDale
|
908ff753a1
fix: phi_3.5_v loading (#896)
|
hai 3 semanas |
|
AlpinDale
|
22a4cd4595
core: fix spec decode metrics and envs circular import (#889)
|
hai 3 semanas |
|
AlpinDale
|
901900854e
chore: consolidate environment variables within one file (#882)
|
hai 4 semanas |
|
AlpinDale
|
9288a98084
spec decoding: set the draft model ctxlen to target model (#874)
|
hai 1 mes |
|
AlpinDale
|
483c9e6e59
fix: disable awq_marlin override for awq models (#843)
|
hai 1 mes |
|
AlpinDale
|
2f61644f6e
SPMD optimizations (#824)
|
hai 1 mes |
|
AlpinDale
|
0a369f9171
feat: support chunked prefill with LoRA (#823)
|
hai 1 mes |
|
AlpinDale
|
c6c91edab7
ci: update & overhaul test units (#769)
|
hai 1 mes |
|
AlpinDale
|
0256ed236b
feat: windows support (#790)
|
hai 2 meses |
|
AlpinDale
|
7222b84582
feat: ministral support (#776)
|
hai 2 meses |
|
AlpinDale
|
73177656ed
feat: quant_llm support (#755)
|
hai 3 meses |
|
AlpinDale
|
89a2c6dee1
chore: refactor `MultiModalConfig` initialization and profiling (#745)
|
hai 3 meses |
|
AlpinDale
|
d34e083c48
feat: add experts_int8 support (#730)
|
hai 3 meses |
|
AlpinDale
|
b0f262eec1
feat: FP8 quantization support for AMD ROCm (#729)
|
hai 3 meses |
|
AlpinDale
|
28b6397188
chore: quant config for speculative draft models (#719)
|
hai 4 meses |
|
AlpinDale
|
577586309d
chore: multi-step args and sequence modifications (#713)
|
hai 4 meses |
|
AlpinDale
|
0b8b407b6d
feat: support profiling with multiple multi-modal inputs per prompt (#712)
|
hai 4 meses |
|
AlpinDale
|
d5033e12fd
feat: implement mistral tokenizer mode (#711)
|
hai 4 meses |
|
AlpinDale
|
5d37ec1016
suppress tpu import warning (#696)
|
hai 4 meses |
|
AlpinDale
|
4fe371b7fa
fix: allow passing float for GiB arguments (#690)
|
hai 4 meses |
|
AlpinDale
|
bf88c8567e
feat: mamba model support (#674)
|
hai 4 meses |
|
AlpinDale
|
e200775863
feat: enable using fp8 kv and prefix caching with chunked prefill (#668)
|
hai 4 meses |
|
AlpinDale
|
62111fab17
feat: allow serving encoder-decoder models in the API server (#664)
|
hai 4 meses |
|
AlpinDale
|
3f49a55f82
feat: add INT8 W8A16 quant for TPU (#663)
|
hai 4 meses |
|
AlpinDale
|
3405782f24
fix: max_num_batched_tokens should not be limited for lora (#658)
|
hai 4 meses |
|
AlpinDale
|
a0e446a17d
feat: initial encoder-decoder support with BART model (#633)
|
hai 4 meses |
|
AlpinDale
|
f1d0b77c92
[0.6.0] Release Candidate (#481)
|
hai 4 meses |
|
AlpinDale
|
9d81716bfd
[v0.5.3] Release Candidate (#388)
|
hai 8 meses |
|
AlpinDale
|
78d66f16d1
Chunked Prefill Part 1 (#384)
|
hai 9 meses |
