Accueil Explorer Aide
S'inscrire Connexion
david
/
flash-attention
miroir de https://github.com/Dao-AILab/flash-attention
1
0
Fork 0
Fichiers Tickets 0 Wiki
Aborescence: 0dfb281743
Branches Tags
flash-attention
/
training
/
configs
/
experiment
/
owt
/
gpt2m.yaml

gpt2m.yaml 180 B
Historique Raw

1234567891011 
  1. # @package _global_
    2. 

  2. defaults:
    3. 

  3.   - /experiment/owt/gpt2s.yaml
    4. 

  4.   - override /model/gpt2model: gpt2-medium
    5. 

  5. 

  6. datamodule:
    7. 

  7.   batch_size: 8  # Per GPU
    8. 

  8. 

  9. train:
    10. 

  10.   optimizer:
    11. 

  11.     lr: 1.5e-4
    12.