 Jay Shah
|
b3d60fa3a5
prune more dead code
|
vor 3 Monaten |
|
Jay Shah
|
35f3542442
refactor names
|
vor 3 Monaten |
|
Jay Shah
|
eb9c0ee22a
add rmem -> gmem for fp8
|
vor 3 Monaten |
|
Jay Shah
|
7169b23399
unify rmem -> gmem methods
|
vor 3 Monaten |
|
Jay Shah
|
ab5d336e61
better writeout logic with vectorization
|
vor 3 Monaten |
|
Jay Shah
|
d437d3dd5c
remove smem usage for when rmem -> gmem epilogue is used
|
vor 3 Monaten |
 Ganesh Bikshandi
|
a075e769fb
handle gqa_parallel with rmem-to-gmem. Not validating yet.
|
vor 3 Monaten |
|
Jay Shah
|
4a4dbd29c5
move IsRegToGmem
|
vor 3 Monaten |
|
Jay Shah
|
8f45a8cfa2
tests passing now for non-gqa impl
|
vor 3 Monaten |
|
Ganesh Bikshandi
|
f0b49460ec
changes to use tiledcopy (still not passing).
|
vor 3 Monaten |
|
Ganesh Bikshandi
|
8fbefa8ac4
adding rmem to gmem. (Not validating yet).
|
vor 3 Monaten |
|
Jay Shah
|
16eb1e53fd
remove deprecated fp8 code
|
vor 3 Monaten |
|
Jay Shah
|
aa45d75f64
dont write out zero for split kernel, only lse=-inf
|
vor 3 Monaten |
|
Jay Shah
|
2472e5e0b4
add 'in principle' fp8 kv cache support
|
vor 3 Monaten |
|
Jay Shah
|
e36e004cb3
change fp8 code path to allow for split kernel and kv cache without perf regression
|
vor 3 Monaten |
|
Ganesh Bikshandi
|
9c97808349
changes for correct lse write out for splits=1 and splits > 1 case.
|
vor 4 Monaten |
|
Jay Shah
|
68ff3f7bee
add 1 mma warpgroup option, enable splitkv for hdim 256
|
vor 4 Monaten |
|
Jay Shah
|
0a1a0c22b6
refactor for split kv
|
vor 4 Monaten |
|
Jay Shah
|
0c4cea947b
add in lse writeout and store zero
|
vor 4 Monaten |
|
Jay Shah
|
535b8279bb
complete gqa parallel changes for non-causal
|
vor 4 Monaten |
|
Jay Shah
|
d2f049c8bc
change logic of Q loads for gqa parallelization
|
vor 5 Monaten |
|
Jay Shah
|
74f160ba43
change template parameter for SeqLenTraits for ease of further extension
|
vor 5 Monaten |
 Ying Zhang
|
a3a257c71d
Fix out-of-bound writes for var-seq-len zero-length KVs
|
vor 5 Monaten |
 Ying Zhang
|
3669b25206
bwd benchmark + small fixes (#1129)
|
vor 5 Monaten |
|
jayhshah
|
5018ac6ac5
Fp8 kernel with "in-kernel" transpose of V in producer (#1100)
|
vor 5 Monaten |
 Tri Dao
|
3aae9c18c1
Revert "Changes For FP8 (#1075)"
|
vor 5 Monaten |
 ganeshcolfax
|
1899c970c8
Changes For FP8 (#1075)
|
vor 5 Monaten |
|
Ying Zhang
|
dfe1a59e4b
Add var-seq-len to FA3 fp16 / bf16 fwd (#1072)
|
vor 5 Monaten |
|
Tri Dao
|
74b0761ff7
[FA3] BF16 forward
|
vor 6 Monaten |
|
Tri Dao
|
7f67966cc7
FA3 initial code release
|
vor 6 Monaten |
