 AlpinDale
|
b02e633691
configure endpoint
|
vor 6 Monaten |
|
AlpinDale
|
ed759f065d
chore: tokenizer_revision -> revision
|
vor 6 Monaten |
|
AlpinDale
|
2e0b115ce1
move func tracing to utils
|
vor 6 Monaten |
|
AlpinDale
|
41338053e7
feat: add shutdown method to engine
|
vor 6 Monaten |
|
AlpinDale
|
199e776722
chore: move ray utils to executor dir
|
vor 6 Monaten |
|
AlpinDale
|
e7b1368156
feat: Phi3 support
|
vor 6 Monaten |
|
AlpinDale
|
1225c4dfd6
fix: illegal mem access crash for marlin
|
vor 6 Monaten |
|
AlpinDale
|
d1a3c7bc2c
chore: simplify try-finally logic in pynccl
|
vor 6 Monaten |
|
AlpinDale
|
440384d776
chore: use nvidia-ml-py instead of pynvml
|
vor 6 Monaten |
|
AlpinDale
|
46159b107a
formatting: pt1
|
vor 6 Monaten |
|
AlpinDale
|
4c746d8baa
chore: init nccl using the gloo backend
|
vor 6 Monaten |
|
AlpinDale
|
bf2dd2bee9
feat: allow multiple sampling params in LLM class
|
vor 6 Monaten |
 Orion
|
a2a24e9b0d
feat: list support in message.content (#503)
|
vor 6 Monaten |
 Bruno Renié
|
9c45fe9a2a
openai: fix metrics endpoint (#512)
|
vor 6 Monaten |
 AlpinDale
|
fca911ee0a
vLLM Upstream Sync (#526)
|
vor 6 Monaten |
|
AlpinDale
|
8be299e78b
fix: lora load check
|
vor 7 Monaten |
|
AlpinDale
|
096d9eb6c5
enhance nvlink detection
|
vor 7 Monaten |
|
AlpinDale
|
fb7825df8f
squash logprobs
|
vor 7 Monaten |
|
AlpinDale
|
66b7bc4415
sliding window in prefix kernel
|
vor 7 Monaten |
|
AlpinDale
|
42998e423c
better quant verification
|
vor 7 Monaten |
|
AlpinDale
|
483c95a2f8
fix ops in gptq and awq
|
vor 7 Monaten |
|
AlpinDale
|
8f9cb7235c
chore: allow multiple served model names
|
vor 7 Monaten |
|
AlpinDale
|
fc80f57967
fix: correct file name for qwen2 moe
|
vor 7 Monaten |
|
AlpinDale
|
f894f7b176
Revert "reduce dedupe by wrapping in general worker class"
|
vor 8 Monaten |
|
AlpinDale
|
082b0b03bc
Revert "actually run the workers"
|
vor 8 Monaten |
|
AlpinDale
|
36cf32649d
actually run the workers
|
vor 8 Monaten |
|
AlpinDale
|
9fff6fb507
reduce dedupe by wrapping in general worker class
|
vor 8 Monaten |
|
AlpinDale
|
b92bddafe9
time.monotonic() -> time.time()
|
vor 8 Monaten |
|
AlpinDale
|
0178b4d976
docker: add AWS Neuron Docker image
|
vor 8 Monaten |
|
AlpinDale
|
949f0445de
readme: update installation command
|
vor 8 Monaten |
