 AlpinDale
|
224280ad45
Merge branch 'main' into multi-step
|
vor 1 Monat |
|
AlpinDale
|
2242dc7965
formatting
|
vor 1 Monat |
|
AlpinDale
|
2242841380
add to benchmark
|
vor 1 Monat |
|
AlpinDale
|
2242cb25dc
fix: unbound tokenizer error
|
vor 1 Monat |
|
AlpinDale
|
224290913a
remove kv cache estimation
|
vor 1 Monat |
|
AlpinDale
|
22420ca5d0
add tests
|
vor 1 Monat |
|
AlpinDale
|
2242a70c3c
patch gpu executors
|
vor 1 Monat |
|
AlpinDale
|
2242e966b7
async engine impl
|
vor 1 Monat |
|
AlpinDale
|
2242981764
switch to cpu from numpy for sampled token ids
|
vor 1 Monat |
|
AlpinDale
|
224262f125
broadcastable model input in worker base
|
vor 1 Monat |
|
AlpinDale
|
2242a835e7
add multistep worker
|
vor 1 Monat |
|
AlpinDale
|
22428c9911
add multistep model runner
|
vor 1 Monat |
|
AlpinDale
|
2242284b19
add broadcastable model input base
|
vor 1 Monat |
 AlpinDale
|
3d83e64f8e
feat: add metrics for prefix cache hit rate (#829)
|
vor 1 Monat |
|
AlpinDale
|
22425b689d
fix: XPU build
|
vor 1 Monat |
|
AlpinDale
|
bfc8988116
feat: add cuda sampling kernels for top_k and top_p (#828)
|
vor 1 Monat |
|
AlpinDale
|
22427602eb
feat: add top-nsigma sampling method
|
vor 1 Monat |
|
AlpinDale
|
22429e4a10
fix: sampler test with new transformers version
|
vor 1 Monat |
|
AlpinDale
|
2f61644f6e
SPMD optimizations (#824)
|
vor 1 Monat |
|
AlpinDale
|
32a37e8107
tests: partially fix tensorizer and logprobs tests
|
vor 1 Monat |
|
AlpinDale
|
7f1c9af5e2
fix: fp8 quant test
|
vor 1 Monat |
|
AlpinDale
|
173ac23399
fix: experts int8 quant test
|
vor 1 Monat |
|
AlpinDale
|
68f050129d
fix: lora worker manager test import
|
vor 1 Monat |
|
AlpinDale
|
3661de812d
fix: lora layer test
|
vor 1 Monat |
|
AlpinDale
|
0a369f9171
feat: support chunked prefill with LoRA (#823)
|
vor 1 Monat |
|
AlpinDale
|
e5b1afe625
feat: add chat method for LLM class (#822)
|
vor 1 Monat |
|
AlpinDale
|
262cbc63b7
fix: vision api test template path
|
vor 1 Monat |
|
AlpinDale
|
b0113a1eaa
fix: tokenization api test (#821)
|
vor 1 Monat |
|
AlpinDale
|
c6c91edab7
ci: update & overhaul test units (#769)
|
vor 1 Monat |
|
AlpinDale
|
f088ea81c7
fix: --max-seq-len-to-capture arg (#818)
|
vor 1 Monat |
