 AlpinDale
|
9d7beaa5b9
chore: separate kv_scale into k_scale and v_scale
|
vor 5 Monaten |
 AlpinDale
|
156f577f79
feat: switch from `PYBIND11_MODULE` to `TORCH_LIBRARY` (#569)
|
vor 5 Monaten |
|
AlpinDale
|
3bdeb3e116
fix: clang formatting for all kernels (#558)
|
vor 6 Monaten |
|
AlpinDale
|
251568470e
initial nvidia fp8 e4m3 for kv cache
|
vor 6 Monaten |
|
AlpinDale
|
8b56dc4347
dict -> torch.Tensor for blocks_to_swap
|
vor 6 Monaten |
|
AlpinDale
|
21ce19b3ea
blocks_to_copy dict -> torch.Tensor
|
vor 6 Monaten |
|
AlpinDale
|
2351a0e2cd
feat: FlashInfer backend for decoding phase (#548)
|
vor 6 Monaten |
|
AlpinDale
|
9d81716bfd
[v0.5.3] Release Candidate (#388)
|
vor 8 Monaten |
|
AlpinDale
|
f8dfac6372
chore: attention refactor and upstream sync apr01 (#365)
|
vor 9 Monaten |
|
AlpinDale
|
9810daa699
feat: INT8 KV Cache (#298)
|
vor 10 Monaten |
|
AlpinDale
|
8e1cd54497
fix: do not include fp8 for rocm (#271)
|
vor 11 Monaten |
|
AlpinDale
|
31c95011a6
feat: FP8 E5M2 KV Cache (#226)
|
vor 1 Jahr |
|
AlpinDale
|
8fa608aeb7
feat: replace Ray with NCCL for control plane comms (#221)
|
vor 1 Jahr |
|
AlpinDale
|
15a0454172
feat: FP8 KV Cache (#185)
|
vor 1 Jahr |
|
AlpinDale
|
b9b295d74e
chore: backlogs 1 (#191)
|
vor 1 Jahr |
|
AlpinDale
|
7d91e9e0f2
feat: CUDA graphs (#172)
|
vor 1 Jahr |
|
AlpinDale
|
1334a833a4
feat: AMD ROCm support (#95)
|
vor 1 Jahr |
|
AlpinDale
|
3d72f05c7b
feat: flattened 1D tensor -> 2D tensor (#85)
|
vor 1 Jahr |
|
AlpinDale
|
32844c1522
add GELU kernels and remove compile bloat
|
vor 1 Jahr |
|
AlpinDale
|
76b2e4a445
Merge dev branch into main (#7)
|
vor 1 Jahr |
|
AlpinDale
|
081545bde6
fix: various CUDA kernel tweaks
|
vor 1 Jahr |
|
AlpinDale
|
b8f4337c5b
chore: various fixes
|
vor 1 Jahr |
|
AlpinDale
|
a409431c40
feat: draft for cuda kernels
|
vor 1 Jahr |
