 Tri Dao
|
ccbb14f38e
Implement rotary embedding in flash_attn_with_kvcache
|
vor 1 Jahr |
|
Tri Dao
|
a157cc8c9b
[FT] Implement MQA/GQA
|
vor 1 Jahr |
|
Tri Dao
|
2800efc71f
[FT] rotary_cos/sin should have batch_size dimension
|
vor 1 Jahr |
|
Tri Dao
|
3a9bfd076f
[FT] rotary_cos/sin should have shape (dim) instead of (seqlen, dim)
|
vor 1 Jahr |
|
Tri Dao
|
62e9814466
[Rotary] Make sure frequency calculation is in fp32
|
vor 1 Jahr |
|
Tri Dao
|
48bc6eacd6
[Gen] Add rotary base as an argument to FT attention kernel
|
vor 1 Jahr |
|
Tri Dao
|
f1e01c27ba
[Gen] Pass qkv_stride to ft_attention kernel for batched generation
|
vor 1 Jahr |
|
Tri Dao
|
7c2191542a
[Gen] Make generation work with Tensor Parallel
|
vor 1 Jahr |
|
Tri Dao
|
a01d1213d7
[Gen] Add kernel from FasterTransformer for benchmarking
|
vor 1 Jahr |
