 AlpinDale
|
9d81716bfd
[v0.5.3] Release Candidate (#388)
|
hai 8 meses |
|
AlpinDale
|
feb5840f2a
feat: async tokenization (#374)
|
hai 10 meses |
|
AlpinDale
|
0f6d56b07f
feat: model executor refactor (#367)
|
hai 10 meses |
|
AlpinDale
|
b361096463
fix: tokenizer when using ray (#366)
|
hai 10 meses |
|
AlpinDale
|
f8dfac6372
chore: attention refactor and upstream sync apr01 (#365)
|
hai 10 meses |
|
AlpinDale
|
e42a78381a
feat: switch from pylint to ruff (#322)
|
hai 10 meses |
|
AlpinDale
|
c2d77b1822
chore: logging refactor (#302)
|
hai 10 meses |
|
AlpinDale
|
9810daa699
feat: INT8 KV Cache (#298)
|
hai 10 meses |
|
AlpinDale
|
2d3d44b3e9
chore: add health check for ray workers (#290)
|
hai 10 meses |
|
AlpinDale
|
ac82b67f75
feat: naive context shift and various QoL changes (#289)
|
hai 10 meses |
|
AlpinDale
|
657aec0cbd
refactor: OpenAI endpoint (#261)
|
hai 11 meses |
|
AlpinDale
|
4b80b42362
fix: memory leaks due to nccl cuda graphs (#275)
|
hai 11 meses |
|
AlpinDale
|
2c08aa5af4
chore: remove eos token from output (#272)
|
hai 11 meses |
|
AlpinDale
|
ea0f57b233
feat: allow further support for non-cuda devices (#247)
|
hai 11 meses |
|
AlpinDale
|
d2db4143fa
feat: add grafana for metrics (#240)
|
hai 1 ano |
|
AlpinDale
|
31c95011a6
feat: FP8 E5M2 KV Cache (#226)
|
hai 1 ano |
|
AlpinDale
|
641bb0f6e9
feat: add custom allreduce kernels (#224)
|
hai 1 ano |
|
AlpinDale
|
c0aac15421
feat: S-LoRA support (#222)
|
hai 1 ano |
|
AlpinDale
|
8fa608aeb7
feat: replace Ray with NCCL for control plane comms (#221)
|
hai 1 ano |
|
AlpinDale
|
15a0454172
feat: FP8 KV Cache (#185)
|
hai 1 ano |
|
AlpinDale
|
b9b295d74e
chore: backlogs 1 (#191)
|
hai 1 ano |
 AlpinDale
|
17cdc5ac23
yapf
|
hai 1 ano |
 KaraKaraWitch
|
9a0b5a197d
fix: set CPU Affinity (#187)
|
hai 1 ano |
|
AlpinDale
|
7d91e9e0f2
feat: CUDA graphs (#172)
|
hai 1 ano |
|
AlpinDale
|
6c50f5b067
chore: include stop strings in output (#168)
|
hai 1 ano |
|
AlpinDale
|
980673ffb7
fix: fractional gpus (#157)
|
hai 1 ano |
|
AlpinDale
|
81e7981dce
feat: add prometheus production metrics (#154)
|
hai 1 ano |
|
AlpinDale
|
653da510d1
chore: rewrite InputMetadata (#143)
|
hai 1 ano |
|
AlpinDale
|
6c914ea0e4
fix: `SequenceOutputs` -> `SequenceOutput` (#133)
|
hai 1 ano |
|
AlpinDale
|
1aab8a7d6f
feat: speedup compilation times by 3x (#130)
|
hai 1 ano |
