 AlpinDale
|
b6c4dfce23
chore: refactor TPU model runner and worker
|
hace 4 meses |
 AlpinDale
|
1ff6d4c3d7
feat: support pipeline parallel on indivisible GPU count (#587)
|
hace 4 meses |
|
AlpinDale
|
4f7d212b70
feat: remove vision language config
|
hace 4 meses |
|
AlpinDale
|
d0ff3fd59e
fix: tpu sampler output
|
hace 5 meses |
|
AlpinDale
|
d2461161ec
chore: optimize KV cache swapping for TPU
|
hace 5 meses |
|
AlpinDale
|
8b626e4032
fix: cpu kv cache allocation for TPU
|
hace 5 meses |
|
AlpinDale
|
fcd58614f4
feat: support parallel sampling and swapping in TPU
|
hace 5 meses |
|
AlpinDale
|
af1286f9fa
fix: kv cache size calculation on TPUs
|
hace 5 meses |
|
AlpinDale
|
608e8e1310
chore: refactor TPU backend to make it more similar to GPU backend
|
hace 5 meses |
|
AlpinDale
|
a524667db0
fix: device assertion for sdpa backend; fix env for tpu worker
|
hace 5 meses |
|
AlpinDale
|
fe21123a1c
feat: TPU support (#570)
|
hace 5 meses |
