 AlpinDale
|
60ca1e1e5e
feat: add ngram prompt lookup decoding for speculative decoding (#438)
|
vor 9 Monaten |
|
AlpinDale
|
d8c4193704
feat: Speculative Decoding using a draft model (#432)
|
vor 9 Monaten |
 AlpinDale
|
8d26cf3876
simplify model_executor logic
|
vor 9 Monaten |
|
AlpinDale
|
4d33ce60da
feat: Triton flash attention backend for ROCm (#407)
|
vor 9 Monaten |
|
AlpinDale
|
9aaeb5d349
add speculative config and arg for later
|
vor 9 Monaten |
|
AlpinDale
|
753f6dc51b
add v2 block manager
|
vor 9 Monaten |
|
AlpinDale
|
7b9c08afae
vision model support
|
vor 9 Monaten |
|
AlpinDale
|
d1786645a3
fix formatting
|
vor 9 Monaten |
|
AlpinDale
|
2319b411ce
refactor: neuron support
|
vor 9 Monaten |
|
AlpinDale
|
0f6d56b07f
feat: model executor refactor (#367)
|
vor 9 Monaten |
|
AlpinDale
|
f8dfac6372
chore: attention refactor and upstream sync apr01 (#365)
|
vor 9 Monaten |
