david/flash-attention

mirror of https://github.com/Dao-AILab/flash-attention

Author	SHA1 Message	Date
Tri Dao	3a9bfd076f [FT] rotary_cos/sin should have shape (dim) instead of (seqlen, dim)	1 year ago
Tri Dao	62e9814466 [Rotary] Make sure frequency calculation is in fp32	1 year ago
Tri Dao	48bc6eacd6 [Gen] Add rotary base as an argument to FT attention kernel	1 year ago
Tri Dao	f5d0fbd468 [FT] Fix FT's single query attention for bf16 hdim128 rotary	1 year ago
Tri Dao	a01d1213d7 [Gen] Add kernel from FasterTransformer for benchmarking	1 year ago