sclarkson 1feb711f46 Fix compilation with clang on ARM64 (#1285) | vor 1 Woche | |
---|---|---|
.. | ||
README.md | vor 2 Jahren | |
fused_dense.cpp | vor 1 Woche | |
fused_dense_cuda.cu | vor 1 Jahr | |
setup.py | vor 9 Monaten |
This CUDA extension implements fused matmul + bias (forward and backward), and fused matmul + bias + gelu (forward and backward), adapted from Apex's FusedDense. We make it work for bfloat16.
For best performance, you should use CUDA >= 11.8. CuBLAS versions before this doesn't have the best matmul + bias + gelu performance for bfloat16.
It has only been tested on A100s.
cd csrc/fused_dense_lib && pip install .