 Tri Dao
|
c7f32a8409
[CrossEntropy] Support precomputed LSE
|
3 months ago |
|
Tri Dao
|
08124c8f9c
[CrossEntropy] Implement logit_scale option
|
1 year ago |
|
Tri Dao
|
aaa1474129
[CrossEntropy] Simplify the case of large vocab with Tensor Parallel
|
1 year ago |
|
Tri Dao
|
5400fdc4ac
[CE] Implement CrossEntropyLoss in Triton
|
1 year ago |
|
Tri Dao
|
0e8c46ae08
Run isort and black on test files
|
1 year ago |
|
Tri Dao
|
c6ecd40a59
Tweak CrossEntropyLoss to take process_group in init
|
1 year ago |
|
Tri Dao
|
b4018a5028
Implement Tensor Parallel for GPT model
|
2 years ago |
|
Tri Dao
|
dff68c2b22
Add smoothing for CrossEntropyParallel, rename to CrossEntropyLoss
|
2 years ago |
|
Tri Dao
|
343492ec30
Make nccl operations async in CrossEntropyLossParallel
|
2 years ago |
|
Tri Dao
|
7c9953815a
Add fused cross entropy loss
|
2 years ago |
