 AlpinDale
|
81c28d2a7f
fix: use nvml to get consistent device names (#739)
|
4 months ago |
|
AlpinDale
|
5559c5886f
fix: clear engine ref in RPC server (#738)
|
4 months ago |
|
AlpinDale
|
ef3a0f4cb1
fix: `custom_ar` check (#737)
|
4 months ago |
|
AlpinDale
|
ccbda97416
fix: types in AQLM and GGUF for dynamo support (#736)
|
4 months ago |
|
AlpinDale
|
9296d4b25d
feat: dynamo support for ScalarType (#733)
|
4 months ago |
|
AlpinDale
|
d9d85eeb6e
chore: register lora functions as torch ops (#732)
|
4 months ago |
|
AlpinDale
|
7a313483f1
chore: move update_flash_attn_metadata to attn backend (#731)
|
4 months ago |
|
AlpinDale
|
d34e083c48
feat: add experts_int8 support (#730)
|
4 months ago |
|
AlpinDale
|
b0f262eec1
feat: FP8 quantization support for AMD ROCm (#729)
|
4 months ago |
|
AlpinDale
|
c744443679
ci: bump to 0.6.1.post1 (#728)
|
4 months ago |
 miku448
|
9c0e7d95c8
fix: libcudart path for some versions of pytorch (#726)
|
4 months ago |
|
AlpinDale
|
4648f16c84
chore: fix return statement in Detokenizer class (#727)
|
4 months ago |
|
AlpinDale
|
a286adaeaa
feat: launch API server with uvloop (#725)
|
4 months ago |
|
AlpinDale
|
60b702a827
chore: register custom torch ops for flash-attn and flashinfer (#724)
|
4 months ago |
|
AlpinDale
|
8e0d376f1c
ci: bump aphrodite to 0.6.1 (#722)
|
4 months ago |
|
AlpinDale
|
12e40ae6fd
chore: update grafana template (#721)
|
4 months ago |
|
AlpinDale
|
61c7182491
feat: enable prompt logprobs in OpenAI API (#720)
|
4 months ago |
|
AlpinDale
|
28b6397188
chore: quant config for speculative draft models (#719)
|
4 months ago |
|
AlpinDale
|
8e22069c9e
fix: weight loading for scalars (#718)
|
4 months ago |
|
AlpinDale
|
d289c3855b
fix: install protobuf for cpu (#716)
|
4 months ago |
|
AlpinDale
|
008e646c7e
chore: add support for up to 2048 block size (#715)
|
4 months ago |
|
AlpinDale
|
1c519cc6ac
chore: set per-rank XLA cache for TPU (#714)
|
4 months ago |
|
AlpinDale
|
577586309d
chore: multi-step args and sequence modifications (#713)
|
4 months ago |
|
AlpinDale
|
0b8b407b6d
feat: support profiling with multiple multi-modal inputs per prompt (#712)
|
4 months ago |
|
AlpinDale
|
d5033e12fd
feat: implement mistral tokenizer mode (#711)
|
4 months ago |
|
AlpinDale
|
ebf01d665b
fix: disable embeddings API for chat models (#710)
|
4 months ago |
|
AlpinDale
|
198029295c
fix: empty sampler output when temperature is too low (#709)
|
4 months ago |
|
AlpinDale
|
8878e3a63f
fix: import ray under a guard (#708)
|
4 months ago |
|
AlpinDale
|
36241f98af
feat: add support for multi-host tpu (#707)
|
4 months ago |
|
AlpinDale
|
04da8c33bd
Revert "chore: use the `compressed-tensors` library to avoid code reuse (#704)" (#706)
|
4 months ago |
