 Tri Dao
|
abbc131173
[LayerNorm] Switch from CUDA to Triton implementation
|
11 mēneši atpakaļ |
|
Tri Dao
|
393882bc08
[LayerNorm] Implement LN with parallel residual, support dim 8k
|
1 gadu atpakaļ |
|
Tri Dao
|
6738d9477d
[LayerNorm] Implement RMS Norm
|
1 gadu atpakaļ |
|
Tri Dao
|
8c6609ae1a
[LayerNorm] Support all dimensions up to 6k (if divisible by 8)
|
2 gadi atpakaļ |
|
Tri Dao
|
0bf5e50038
Release training code
|
2 gadi atpakaļ |
|
Tri Dao
|
43ab0b5205
Mention that some CUDA extensions have only been tested on A100s
|
2 gadi atpakaļ |
|
Tri Dao
|
2e33fc8e36
Add GPT and ViT models
|
2 gadi atpakaļ |
|
Tri Dao
|
fa6d1ce44f
Add fused_dense and dropout_add_layernorm CUDA extensions
|
2 gadi atpakaļ |
