 AlpinDale
|
65cd99ba89
fix KVCache type
|
vor 8 Monaten |
|
AlpinDale
|
9e52445ba0
formatting
|
vor 8 Monaten |
|
AlpinDale
|
6c43e00e60
add jamba modeling code
|
vor 8 Monaten |
|
AlpinDale
|
4fbb052b34
add jamba config file
|
vor 8 Monaten |
|
AlpinDale
|
a1f18f17e6
modify the cache engine and model runner/worker to support mamba states
|
vor 8 Monaten |
|
AlpinDale
|
f60803384d
move out of ops dir
|
vor 8 Monaten |
|
AlpinDale
|
2ced01bc3e
clean up interfaces and add selective state update triton kernels
|
vor 8 Monaten |
|
AlpinDale
|
7fd3bd4bf2
add selective scan kernels
|
vor 8 Monaten |
|
AlpinDale
|
44b02e94cd
add forward kernels for causal depthwise conv1d
|
vor 8 Monaten |
 sgsdxzy
|
f3b546e33a
feat: upport twe lm_head for quantized weights (#409)
|
vor 8 Monaten |
|
sgsdxzy
|
214151b04c
fix: max_num_batched_tokens for chunked_prefill (#412)
|
vor 8 Monaten |
|
AlpinDale
|
1dccb03b17
incorrect comparison for hadamard and punica checks
|
vor 8 Monaten |
|
sgsdxzy
|
6a0a6360f1
fix: Allow setting config-path when converting ggufs. (#410)
|
vor 8 Monaten |
|
sgsdxzy
|
fcfb72af24
Support arbitrary model in GGUF. (#381)
|
vor 8 Monaten |
 AlpinDale
|
bd0ddf1cfe
feat: EETQ quantization (#408)
|
vor 8 Monaten |
|
AlpinDale
|
b1caee23a6
cache the p2p access check for memory saving
|
vor 8 Monaten |
|
AlpinDale
|
373e0d3c01
fix neuron
|
vor 8 Monaten |
|
AlpinDale
|
28bcca2396
incorrect use of monotonic time in metrics logger
|
vor 8 Monaten |
|
AlpinDale
|
4ba273886a
debug logging for distributed_init_method
|
vor 8 Monaten |
|
AlpinDale
|
1270b5567e
triton compile error for flash_attn
|
vor 8 Monaten |
|
AlpinDale
|
f375353026
enable custom_all_reduce by default in llm.py
|
vor 8 Monaten |
|
AlpinDale
|
2d2b43fe00
fix type hint
|
vor 8 Monaten |
|
AlpinDale
|
531969a0b2
move merge_async_iterators to common utils
|
vor 8 Monaten |
|
AlpinDale
|
c18bf116da
fix stop strings not being excluded from outputs
|
vor 8 Monaten |
|
AlpinDale
|
5ab7a159d7
fix formatting for previous commit
|
vor 8 Monaten |
|
AlpinDale
|
b6bbf584ac
fix echo
|
vor 8 Monaten |
|
AlpinDale
|
6e0761ba5d
make init_distributed_environment compatible with init_process_group
|
vor 8 Monaten |
|
AlpinDale
|
083ba7b452
roll back chunked prefill changes to SDPA, isolate cpu worker
|
vor 8 Monaten |
|
AlpinDale
|
8c67b37131
fix docstrings
|
vor 8 Monaten |
|
AlpinDale
|
fe17712f29
fully working chunked prefill
|
vor 8 Monaten |
