 AlpinDale
|
ae04f57ec1
feat: Pipeline Parallel support (#581)
|
vor 6 Monaten |
 AlpinDale
|
fad45609b8
chore: remove logical token blocks (turns out they are not needed)
|
vor 6 Monaten |
|
AlpinDale
|
237fa59aea
feat: support CPU/GPU swapping in BlockManagerV2
|
vor 6 Monaten |
|
AlpinDale
|
9099040472
feat: cross-attention kv caching support
|
vor 6 Monaten |
|
AlpinDale
|
8b56dc4347
dict -> torch.Tensor for blocks_to_swap
|
vor 6 Monaten |
|
AlpinDale
|
148aca8ff1
cow => dict[int, list] -> list
|
vor 6 Monaten |
|
AlpinDale
|
25c2b6feca
ignore infeasible swap requests
|
vor 6 Monaten |
|
AlpinDale
|
6f6bf568e5
enable prefix caching with v2 block manager for spec decoding
|
vor 6 Monaten |
|
AlpinDale
|
fca911ee0a
vLLM Upstream Sync (#526)
|
vor 7 Monaten |
|
AlpinDale
|
9d81716bfd
[v0.5.3] Release Candidate (#388)
|
vor 9 Monaten |
