 Tri Dao
|
abbc131173
[LayerNorm] Switch from CUDA to Triton implementation
|
1 year ago |
|
Tri Dao
|
f1a73d0740
Run isort and black on python files
|
1 year ago |
|
Tri Dao
|
ada4710d70
[ViT] Run black on vit.py
|
1 year ago |
|
Tri Dao
|
a81900d4c1
[ViT] Minor fix so it runs
|
1 year ago |
|
Tri Dao
|
88173a1aaf
[FusedDense] Support relu, rename FusedDenseGeluDense -> FusedMLP
|
2 years ago |
|
Tri Dao
|
780e8eeabb
[ViT] Support timm checkpoint, add tests
|
2 years ago |
|
Tri Dao
|
ef085cfcda
[ViT] Fix extra norm_0, use new LN order in Block
|
2 years ago |
|
Tri Dao
|
1feb94265c
[ViT] Use dropout_add_ln for the 1st layer norm
|
2 years ago |
|
Tri Dao
|
2e33fc8e36
Add GPT and ViT models
|
2 years ago |
