 AlpinDale
|
d0ff3fd59e
fix: tpu sampler output
|
7 months ago |
|
AlpinDale
|
d2461161ec
chore: optimize KV cache swapping for TPU
|
7 months ago |
|
AlpinDale
|
8b626e4032
fix: cpu kv cache allocation for TPU
|
7 months ago |
|
AlpinDale
|
fcd58614f4
feat: support parallel sampling and swapping in TPU
|
7 months ago |
|
AlpinDale
|
af1286f9fa
fix: kv cache size calculation on TPUs
|
7 months ago |
|
AlpinDale
|
608e8e1310
chore: refactor TPU backend to make it more similar to GPU backend
|
7 months ago |
|
AlpinDale
|
a524667db0
fix: device assertion for sdpa backend; fix env for tpu worker
|
7 months ago |
 AlpinDale
|
fe21123a1c
feat: TPU support (#570)
|
7 months ago |
