| .. |
|
attention
|
c577c31aaa
feat: tree attention
|
vor 8 Monaten |
|
common
|
c577c31aaa
feat: tree attention
|
vor 8 Monaten |
|
distributed
|
b1caee23a6
cache the p2p access check for memory saving
|
vor 8 Monaten |
|
endpoints
|
c577c31aaa
feat: tree attention
|
vor 8 Monaten |
|
engine
|
c577c31aaa
feat: tree attention
|
vor 8 Monaten |
|
executor
|
60ca1e1e5e
feat: add ngram prompt lookup decoding for speculative decoding (#438)
|
vor 8 Monaten |
|
kv_quant
|
e42a78381a
feat: switch from pylint to ruff (#322)
|
vor 10 Monaten |
|
lora
|
fe17712f29
fully working chunked prefill
|
vor 8 Monaten |
|
modeling
|
b28011e86e
fix: shard exl2 weights more evenly between ranks (#437)
|
vor 8 Monaten |
|
processing
|
c577c31aaa
feat: tree attention
|
vor 8 Monaten |
|
spec_decode
|
60ca1e1e5e
feat: add ngram prompt lookup decoding for speculative decoding (#438)
|
vor 8 Monaten |
|
task_handler
|
c577c31aaa
feat: tree attention
|
vor 8 Monaten |
|
transformers_utils
|
58b0616dd3
feat: support sharded ggufs (#420)
|
vor 8 Monaten |
|
__init__.py
|
c2aaaefd57
allow out-of-tree model registry
|
vor 9 Monaten |
|
py.typed
|
1c988a48b2
fix logging and add py.typed
|
vor 1 Jahr |
AlpinDale
c577c31aaa
feat: tree attention