提交历史

作者 SHA1 备注 提交日期
  Tri Dao 4ead9bd7cc [FA3] Varlen forward 4 月之前
  Tri Dao 74b0761ff7 [FA3] BF16 forward 5 月之前
  Tri Dao 898dd4bbf2 Pass seqused_k to _flash_attn_varlen_forward 5 月之前
  Tri Dao 7ef24848cf Add FA3 image 5 月之前
  Tri Dao 7f67966cc7 FA3 initial code release 5 月之前
  Tri Dao b4a9dd6c9c Temporarily switch to cutlass fork for more shapes 5 月之前
  Tri Dao 7551202cb2 Bump to v2.6.1 5 月之前
  Tri Dao 844912dca0 [CI] Switch from CUDA 12.2 to 12.3 5 月之前
  Tri Dao 40e534a7f6 Implement cache_leftpad 5 月之前
  Tri Dao 116b05f9b0 [CI] Compile with pytorch 2.4.0.dev20240514 5 月之前
  Tri Dao da11d1b853 Bump v2.6.0 5 月之前
  Tri Dao d0787acc16 Relax dropout_fraction test 5 月之前
  Tri Dao dca6d89da4 Don't support softcap and dropout at the same time 5 月之前
  Tri Dao 81e01efd4b More typo fixes 5 月之前
  Tri Dao 72e27c6320 Fix typo with softcapping 5 月之前
  Tri Dao 3d41db3e2c Only test backward if there's no softcapping 5 月之前
  Tri Dao 908511b2b6 Split into more .cu files to speed up compilation 5 月之前
  Tri Dao 1d536d7de5 Minor cleanup of softcapping 5 月之前
  Tri Dao beb2bf2a32 Drop support for pytorch 1.12, 1.13, and python 3.7 5 月之前
  Phil Wang f4628b43ec missing commas and backwards return arguments (#1032) 5 月之前
  Nicolas Patry 8f873cc6ac Implement softcapping. (#1025) 5 月之前
  Jianwei Dong 4e8d60069f Add the return_softmax_lse parameter to the flash_attn_with_kvcache function to allow returning the logsumexp of the attention scores. (#989) 5 月之前
  muoshuosha 6df7e0a02e Fix the varlen deterministic test (#1023) 5 月之前
  66RING 9486635c92 Fix typos of comments about shape. (#837) 5 月之前
  JDKWangGuan 0d810cfb73 Fix KeyError handling for non-existing key in state_dict.pop() (#898) 5 月之前
  cao lei 6a2a16e994 fix typo (#974) 5 月之前
  Nicolas Patry 5bf201966a Fixing argument checking when using `seqlenq_ngroups_swapped`. (#976) 5 月之前
  Liang ab59ec3590 remove swizzle part of `sV.data()` to get a completely non-swizzle `sVtNoSwizzle` (#984) 5 月之前
  Grigory Sizov f816dee63c Support unpadded LSE layout (#970) 5 月之前
  Tri Dao 320fb59487 Update citation 6 月之前