 sgsdxzy
|
1528ce50e5
fix: abort requests when the connection to /v1/completions is interrupted (#431)
|
8 月之前 |
 AlpinDale
|
531969a0b2
move merge_async_iterators to common utils
|
9 月之前 |
|
AlpinDale
|
083ba7b452
roll back chunked prefill changes to SDPA, isolate cpu worker
|
9 月之前 |
|
AlpinDale
|
8d737c8a9a
fix types in merge_dict
|
9 月之前 |
 AlpinDale
|
071269e406
feat: FP8 E4M3 KV Cache (#405)
|
9 月之前 |
|
AlpinDale
|
14f39af8b5
add dict merging util
|
9 月之前 |
|
AlpinDale
|
6f1d13d30a
better recognize cpu build
|
9 月之前 |
|
AlpinDale
|
a304f76d89
feat: Intel CPU support (#403)
|
9 月之前 |
|
AlpinDale
|
753f6dc51b
add v2 block manager
|
9 月之前 |
|
AlpinDale
|
73890c29e2
ipv6 fix
|
9 月之前 |
|
AlpinDale
|
7b9c08afae
vision model support
|
9 月之前 |
|
AlpinDale
|
2319b411ce
refactor: neuron support
|
9 月之前 |
|
AlpinDale
|
e3252edd07
fix: remove event and stream, add typing (#382)
|
9 月之前 |
|
AlpinDale
|
33b3786175
fix: cache neuron checks (#379)
|
9 月之前 |
|
AlpinDale
|
f8dfac6372
chore: attention refactor and upstream sync apr01 (#365)
|
9 月之前 |
|
AlpinDale
|
e53842bd5d
fix: cuda home detection for fp8 kv cache
|
10 月之前 |
|
AlpinDale
|
e42a78381a
feat: switch from pylint to ruff (#322)
|
10 月之前 |
|
AlpinDale
|
c2d77b1822
chore: logging refactor (#302)
|
10 月之前 |
|
AlpinDale
|
9810daa699
feat: INT8 KV Cache (#298)
|
10 月之前 |
|
AlpinDale
|
ea0f57b233
feat: allow further support for non-cuda devices (#247)
|
11 月之前 |
|
AlpinDale
|
31c95011a6
feat: FP8 E5M2 KV Cache (#226)
|
11 月之前 |
|
AlpinDale
|
c0aac15421
feat: S-LoRA support (#222)
|
11 月之前 |
|
AlpinDale
|
8fa608aeb7
feat: replace Ray with NCCL for control plane comms (#221)
|
11 月之前 |
|
AlpinDale
|
f013d714c0
chore: merge dev branch into main (#177)
|
1 年之前 |
|
AlpinDale
|
2755a48d51
merge dev branch into main (#153)
|
1 年之前 |
 g4rg
|
28fa4ae213
fix: cpu memory limit detection for containers (#103)
|
1 年之前 |
|
AlpinDale
|
efc6f7fbec
chore: reformats (#90)
|
1 年之前 |
|
AlpinDale
|
75c27d3e65
massive overhaul
|
1 年之前 |
|
AlpinDale
|
b8f4337c5b
chore: various fixes
|
1 年之前 |
|
AlpinDale
|
fefbf029c9
revert previous commit
|
1 年之前 |
