 AlpinDale
|
0f4a9ee77b
quantized lm_head (#582)
|
6 hónapja |
 AlpinDale
|
cf472315cc
refactor: isolate FP8 from mixtral
|
6 hónapja |
|
AlpinDale
|
ae04f57ec1
feat: Pipeline Parallel support (#581)
|
6 hónapja |
|
AlpinDale
|
dd378ea063
feat: MLPSpeculator with tensor parallel
|
7 hónapja |
|
AlpinDale
|
3a0fdf7b9b
chore: remove `image_input_type` from VLM config
|
7 hónapja |
|
AlpinDale
|
63b735bc2a
chore: optimize v2 block manager to match the performance of v1
|
7 hónapja |
|
AlpinDale
|
de7e6919c0
feat: support tied weights and input scale for MLPSpeculator
|
7 hónapja |
|
AlpinDale
|
9da2448964
fix: ensure worker model loop is always stopped at the right time
|
7 hónapja |
|
AlpinDale
|
ca6b69966d
fix: explicitly end_forward() calls to flashinfer
|
7 hónapja |
|
AlpinDale
|
3b2666314d
fix: add chunking mechanism to fused_moe
|
7 hónapja |
|
AlpinDale
|
70ecdefc4e
fix: Ray ActorDiedError not available on older ray versions
|
7 hónapja |
|
AlpinDale
|
7253e9052d
feat: integrate typical acceptance sampling for spec decoding
|
7 hónapja |
|
AlpinDale
|
7d79c0e726
chore: use nvml query to avoid accidental cuda initialization
|
7 hónapja |
|
AlpinDale
|
ddb3323f94
refactor: have w8a8 compressed tensors use `process_weights_after_load` for fp8
|
7 hónapja |
|
AlpinDale
|
17f7089e26
fix: `get_min_capability` for all quants
|
7 hónapja |
|
AlpinDale
|
0a6db357d8
fix: use safetensor keys instead of adapter_config.json to find unexpected modules
|
7 hónapja |
|
AlpinDale
|
4f87a14998
chore: allow base64 embeddings
|
7 hónapja |
|
AlpinDale
|
aea0b52e52
fix: torchvision version for rocm
|
7 hónapja |
|
AlpinDale
|
4cdc810b1c
fix: minor TP issues with vision models
|
7 hónapja |
|
AlpinDale
|
336eb4dbf8
fix: raise error in moe kernel if it receives more than 65k tokens
|
7 hónapja |
|
AlpinDale
|
bcc60a6555
chore: optimize SequenceStatus.is_finished by switching to IntEnum
|
7 hónapja |
|
AlpinDale
|
7b04361934
fix: support getting `eos_token_id` from the config file
|
7 hónapja |
|
AlpinDale
|
b8a19ba27f
chore: extend aphrodite metrics logging api
|
7 hónapja |
|
AlpinDale
|
301ec7c77d
fix: pad slot id in tpu runner
|
7 hónapja |
|
AlpinDale
|
d0ff3fd59e
fix: tpu sampler output
|
7 hónapja |
|
AlpinDale
|
b6e60143e7
Flashinfer for prefill phase (#580)
|
7 hónapja |
|
AlpinDale
|
7e66e8f899
fix: only add `Attention.kv_scale` if kv cache quant is enabled
|
7 hónapja |
|
AlpinDale
|
bbde979ecd
DeepSeek-V2 (#579)
|
7 hónapja |
|
AlpinDale
|
272c64ab88
chore: allow loading fp8 models with fused qkv/mlp
|
7 hónapja |
|
AlpinDale
|
772a126c08
chore: simplify fp8 weight loading
|
7 hónapja |
