 Joel Lamy-Poirier
|
767b71ccf0
Fix random state for dropout_layer_norm (#315)
|
vor 1 Jahr |
 Ikko Eltociear Ashimine
|
dfc60f6b7d
[LayerNorm] Fix typo in ln_api.cpp
|
vor 1 Jahr |
 Tri Dao
|
393882bc08
[LayerNorm] Implement LN with parallel residual, support dim 8k
|
vor 1 Jahr |
|
Tri Dao
|
eb33e587e9
[LayerNorm] Rename x1 -> residual
|
vor 1 Jahr |
|
Tri Dao
|
6738d9477d
[LayerNorm] Implement RMS Norm
|
vor 1 Jahr |
|
Tri Dao
|
a8cfe51551
Implement Tensor Parallel for transformer Block
|
vor 2 Jahren |
|
Tri Dao
|
5db330519a
[LayerNorm] Support taking subset of input or subset of output
|
vor 2 Jahren |
|
Tri Dao
|
ae137ed17a
[LayerNorm] Fuse LayerScale
|
vor 2 Jahren |
|
Tri Dao
|
8c6609ae1a
[LayerNorm] Support all dimensions up to 6k (if divisible by 8)
|
vor 2 Jahren |
|
Tri Dao
|
fa6d1ce44f
Add fused_dense and dropout_add_layernorm CUDA extensions
|
vor 2 Jahren |
