 JDKWangGuan
|
0d810cfb73
Fix KeyError handling for non-existing key in state_dict.pop() (#898)
|
hai 6 meses |
 Tri Dao
|
ef0ed10622
Add window_size option to MHA and GPT
|
hai 11 meses |
|
Tri Dao
|
abbc131173
[LayerNorm] Switch from CUDA to Triton implementation
|
hai 1 ano |
|
Tri Dao
|
73df3be7d5
Add test for BTLM init
|
hai 1 ano |
|
Tri Dao
|
2e29dacf0c
Implement muParam
|
hai 1 ano |
|
Tri Dao
|
2c7d7b7396
Implement norm head for Baichuan2
|
hai 1 ano |
|
Tri Dao
|
c3b2196652
Add Alibi to MHA, test with Baichuan-13B
|
hai 1 ano |
 Yuchao Dai
|
187c2a0635
Fix E1136 (#563)
|
hai 1 ano |
|
Tri Dao
|
d0032700d1
Add tests for Pythia, GPT-JT, and RedPajama models
|
hai 1 ano |
 Kevin Hu
|
07005806ff
Add BigCode converters (#532)
|
hai 1 ano |
|
Tri Dao
|
798858f9f1
Fix test_baichuan
|
hai 1 ano |
|
Tri Dao
|
7b33743a72
[Gen] Add back num_last_tokens in gpt.py
|
hai 1 ano |
 dan_the_3rd
|
011ec323d6
Support MQA + MP for decoding (#490)
|
hai 1 ano |
|
Tri Dao
|
f8aea6ead0
[GPT] Generalize last_token_only arg to num_last_tokens
|
hai 1 ano |
 Aman Gupta Karmani
|
e0b09891c6
add llama support to GPTPreTrainedModel.from_pretrained (#479)
|
hai 1 ano |
 Xuechen Li
|
25d6b1dbcb
handle uneven heads across ranks when combining state_dicts; resolves #467 (#468)
|
hai 1 ano |
|
Xuechen Li
|
7fcd3e6a04
map custom model state_dict back to huggingface format (#465)
|
hai 1 ano |
|
Tri Dao
|
f1a73d0740
Run isort and black on python files
|
hai 1 ano |
|
Xuechen Li
|
bb4cded17b
support when num_heads is not divisible by world_size; resolves #459 (#461)
|
hai 1 ano |
|
Tri Dao
|
4b661a569d
[GPT] Run black on gpt.py
|
hai 1 ano |
|
Tri Dao
|
184b992dcb
[GPT] Implement parallel LLaMa
|
hai 1 ano |
 Haodong Lyu
|
8ee62efca3
Implement ParallelGatedMlp (#251)
|
hai 1 ano |
|
Tri Dao
|
d38357dd2f
[GPT] Implement Falcon
|
hai 1 ano |
|
Tri Dao
|
425dbcb6c6
[MHA] Implement MQA/GQA
|
hai 1 ano |
|
Tri Dao
|
ec9f74ab9a
[Rotary] Don't store inv_freq in state_dict
|
hai 1 ano |
|
Tri Dao
|
75e334d407
[MLP] Add ParallelMLP
|
hai 1 ano |
|
Tri Dao
|
48bc6eacd6
[Gen] Add rotary base as an argument to FT attention kernel
|
hai 1 ano |
 Federico Berto
|
69f5f7d0a2
[BugFix] cannot unpack non-iterable NoneType object
|
hai 1 ano |
|
Tri Dao
|
a9a4b4e4f2
[LLaMa] Fix last norm layer to use RMSNorm instead of LayerNorm
|
hai 1 ano |
|
Tri Dao
|
ba2fe7f378
[Gen] Move allocate_inference_cache to within the model
|
hai 1 ano |
