 AlpinDale
|
2242ec25d7
gptq activation ordering
|
vor 2 Monaten |
|
AlpinDale
|
679cfcb063
integrate with gptq
|
vor 2 Monaten |
|
AlpinDale
|
2242195d1f
add tests and benchmark scripts
|
vor 2 Monaten |
|
AlpinDale
|
2242f72391
add custom ops for machete
|
vor 2 Monaten |
|
AlpinDale
|
2242de2ad1
fix numeric conversion compilation
|
vor 2 Monaten |
|
AlpinDale
|
2f2ce363e6
lmao!
|
vor 2 Monaten |
|
AlpinDale
|
2242a85d05
fix: cute and torch utils
|
vor 2 Monaten |
|
AlpinDale
|
2242acb4b1
prepack_layout -> prepacked_layout
|
vor 2 Monaten |
|
AlpinDale
|
22423901c8
cmakelists fixes
|
vor 2 Monaten |
|
AlpinDale
|
4dbd02c06e
add machete kernels
|
vor 2 Monaten |
|
AlpinDale
|
224202331c
add cutlass extensions for aphrodite
|
vor 2 Monaten |
|
AlpinDale
|
2242b8cd70
add cute and torch utils
|
vor 2 Monaten |
 AlpinDale
|
563e8f7ac8
fix: latency and serving benchmarks (#841)
|
vor 2 Monaten |
|
AlpinDale
|
7c7ec12f36
chore: refactor executor classes for easier inheritance (#840)
|
vor 2 Monaten |
|
AlpinDale
|
16b587c104
fix: hidden states handling in batch expansion for spec decoding (#839)
|
vor 2 Monaten |
|
AlpinDale
|
60f7b828d5
feat: add skew sampling (#834)
|
vor 2 Monaten |
|
AlpinDale
|
ba9d8f631a
feat: add no_repeat_ngram sampler (#832)
|
vor 2 Monaten |
 Selali
|
4c4a365f77
feat: Add DRY (Don't Repeat Yourself) sampling (#827)
|
vor 2 Monaten |
|
AlpinDale
|
48a8693aed
feat: multi-step scheduling (#831)
|
vor 2 Monaten |
|
AlpinDale
|
2242cb25dc
fix: unbound tokenizer error
|
vor 2 Monaten |
|
AlpinDale
|
3d83e64f8e
feat: add metrics for prefix cache hit rate (#829)
|
vor 2 Monaten |
|
AlpinDale
|
22425b689d
fix: XPU build
|
vor 2 Monaten |
|
AlpinDale
|
bfc8988116
feat: add cuda sampling kernels for top_k and top_p (#828)
|
vor 2 Monaten |
|
AlpinDale
|
22427602eb
feat: add top-nsigma sampling method
|
vor 2 Monaten |
|
AlpinDale
|
22429e4a10
fix: sampler test with new transformers version
|
vor 2 Monaten |
|
AlpinDale
|
2f61644f6e
SPMD optimizations (#824)
|
vor 2 Monaten |
|
AlpinDale
|
32a37e8107
tests: partially fix tensorizer and logprobs tests
|
vor 2 Monaten |
|
AlpinDale
|
7f1c9af5e2
fix: fp8 quant test
|
vor 2 Monaten |
|
AlpinDale
|
173ac23399
fix: experts int8 quant test
|
vor 2 Monaten |
|
AlpinDale
|
68f050129d
fix: lora worker manager test import
|
vor 2 Monaten |
