Home Explore Help
Register Sign In
david
/
flash-attention
mirror of https://github.com/Dao-AILab/flash-attention
1
0
Fork 0
Files Issues 0 Wiki
Tree: 0dfb281743
Branches Tags
flash-attention
/
training
/
configs
/
optimizer
/
adamw-apex-distributed.yaml

adamw-apex-distributed.yaml 123 B
History Raw

123 
  1. # @package train.optimizer
    2. 

  2. _target_: apex.contrib.optimizers.distributed_fused_adam.DistributedFusedAdam
    3. 

  3. adam_w_mode: True
    4.