 AlpinDale
|
64801bbda6
lora: fix fully sharded lora case where all three lora_a could be None
|
hace 3 semanas |
 AlpinDale
|
d69273bd2b
ray: better error when placement group topology is incorrect (#906)
|
hace 3 semanas |
|
AlpinDale
|
6fbab320e7
api: error suppression cleanup + timeout suppression on aborts (#905)
|
hace 3 semanas |
|
AlpinDale
|
ab533e0e60
spec decode: fix logprobs when using speculative decoding (#904)
|
hace 3 semanas |
|
AlpinDale
|
afc9a28aa0
chore: add AphroditeParameter support for FP8 quant (#902)
|
hace 3 semanas |
|
AlpinDale
|
2a60b8f8c9
kernel: do not compile machete for cuda 11 and below (#901)
|
hace 3 semanas |
|
AlpinDale
|
64c05b969a
fix: `ShardedStateLoader` with fp8 quant (#900)
|
hace 3 semanas |
|
AlpinDale
|
132aa2abe4
spec decode: add support for EAGLE (#899)
|
hace 3 semanas |
|
AlpinDale
|
bfc3da41ae
feat: add torch.compile for GemmaRMSNorm (#898)
|
hace 3 semanas |
|
AlpinDale
|
a00ab49e21
api: add client timeouts for the ZeroMQ server (#897)
|
hace 3 semanas |
|
AlpinDale
|
908ff753a1
fix: phi_3.5_v loading (#896)
|
hace 3 semanas |
|
AlpinDale
|
e14223dce5
kernel: use `cub::BlockReduce` instead of custom impl (#895)
|
hace 3 semanas |
|
AlpinDale
|
ff4b7236d5
build: fix invalid path for envs.py in setup (#894)
|
hace 3 semanas |
|
AlpinDale
|
f831fd8312
rocm: fix compile issues with rocm 6.2 (#893)
|
hace 3 semanas |
|
AlpinDale
|
65b71f5fcc
distributed: fix issue for when nodes have multiple network interfaces (#892)
|
hace 3 semanas |
|
AlpinDale
|
653d1a08d4
feat: add support for audio models (#891)
|
hace 3 semanas |
|
AlpinDale
|
22a4cd4595
core: fix spec decode metrics and envs circular import (#889)
|
hace 3 semanas |
|
AlpinDale
|
901900854e
chore: consolidate environment variables within one file (#882)
|
hace 4 semanas |
|
AlpinDale
|
ce6e3d63f7
api: better startup failure UX (#881)
|
hace 4 semanas |
|
AlpinDale
|
db6a50fd5c
async: disable multi-step scheduling for sync engine (#880)
|
hace 4 semanas |
|
AlpinDale
|
afadef06cd
build: pass `PYTHONPATH` from setup.py to cmake (#879)
|
hace 4 semanas |
|
AlpinDale
|
b5aa11020b
api: fix crashes under very high loads (#878)
|
hace 4 semanas |
 Noah Peterson
|
9fd2bfa02e
readme: fix paged attention hyperlink (#876)
|
hace 4 semanas |
|
AlpinDale
|
f797294b29
fix: `add_generation_template` -> `add_generation_prompt` in llm (#877)
|
hace 4 semanas |
|
AlpinDale
|
f0cc35befe
sampler: pad dry sequence breakers tensor (#875)
|
hace 1 mes |
|
AlpinDale
|
9288a98084
spec decoding: set the draft model ctxlen to target model (#874)
|
hace 1 mes |
|
AlpinDale
|
55b7ce56c1
cpu: fix `mm_limits` initialization (#873)
|
hace 1 mes |
|
AlpinDale
|
5bd4473bb6
async: avoid premature exit in the async generator (#872)
|
hace 1 mes |
|
AlpinDale
|
abfd4465ca
feat: add support for chunked prefill + prefix caching (#871)
|
hace 1 mes |
|
AlpinDale
|
ef99a567b6
fix: temp_last warning being repeated for every output token (#869)
|
hace 1 mes |
