 AlpinDale
|
6c1eab6a6c
feat: non-blocking transfer in prepare_input
|
4 maanden geleden |
|
AlpinDale
|
212b9d8a03
refactor: add has_prefix_cache_hit flag to FlashAttentionMetadataBuilder
|
4 maanden geleden |
|
AlpinDale
|
614ca6b0bf
feat: support logits soft capping with flash attention backend
|
4 maanden geleden |
 AlpinDale
|
b15e6376f8
bump to torch 2.4.0, add aphrodite_flash_attn (#614)
|
4 maanden geleden |
|
AlpinDale
|
705e50f4bd
fix: broadcasting logic for multi_modal_kwargs
|
4 maanden geleden |
|
AlpinDale
|
98ad5a6cba
fix: decode tokens w/ CUDA graphs and graps with flashinfer
|
4 maanden geleden |
|
AlpinDale
|
32bdbd1ee4
chore: add fp8 support to `reshape_and_cache_flash`
|
4 maanden geleden |
|
AlpinDale
|
84a9cd25c9
fix: some naming issues
|
4 maanden geleden |
|
AlpinDale
|
51ea8ad376
chore: modularize prepare input and attn metadata builder
|
4 maanden geleden |
|
AlpinDale
|
22305c91e9
refactor _prepare_model_input_tensor and attn metadata builder for most backends
|
4 maanden geleden |
|
AlpinDale
|
9d7beaa5b9
chore: separate kv_scale into k_scale and v_scale
|
4 maanden geleden |
|
AlpinDale
|
2105e4fd6b
feat: correctly invoke prefill & decode kernels for cross-attention
|
4 maanden geleden |
|
AlpinDale
|
405bb74612
Control plane comms refactor (#573)
|
5 maanden geleden |
|
AlpinDale
|
ab7f4ed6e5
chore: revert commit for removing unnecessary copies in flash attn backend
|
5 maanden geleden |
|
AlpinDale
|
156f577f79
feat: switch from `PYBIND11_MODULE` to `TORCH_LIBRARY` (#569)
|
5 maanden geleden |
|
AlpinDale
|
75f97bc25d
bump flash-attn to remove unnecessary copies in the backend
|
5 maanden geleden |
|
AlpinDale
|
696f2cd59c
add phi3_small support with blocksparse attention
|
5 maanden geleden |
|
AlpinDale
|
b8b63eb5ca
fix head_size check for flash attention backend
|
5 maanden geleden |
|
AlpinDale
|
93cffaf446
add flash_attn back
|
5 maanden geleden |
|
AlpinDale
|
0c15965621
fix fp8 kv
|
5 maanden geleden |
|
AlpinDale
|
a94de94c44
refactor: combine the prefill and decode into a single API (#553)
|
5 maanden geleden |
|
AlpinDale
|
01190e5049
use flash attention for the decoding phase
|
5 maanden geleden |
|
AlpinDale
|
50b7c13db0
refactor: attention selector (#552)
|
5 maanden geleden |
|
AlpinDale
|
d11d68f4e6
switch to vllm-flash-attn
|
5 maanden geleden |
|
AlpinDale
|
8b56dc4347
dict -> torch.Tensor for blocks_to_swap
|
5 maanden geleden |
|
AlpinDale
|
3a0d1c7705
add get_name method to attention backends
|
5 maanden geleden |
|
AlpinDale
|
21ce19b3ea
blocks_to_copy dict -> torch.Tensor
|
5 maanden geleden |
|
AlpinDale
|
35ae01d7ba
refactor: attention metadata term
|
5 maanden geleden |
|
AlpinDale
|
aa15d3dc0f
sliding window in prefix prefill kernel
|
5 maanden geleden |
|
AlpinDale
|
fca911ee0a
vLLM Upstream Sync (#526)
|
6 maanden geleden |
