 AlpinDale
|
e13a66925c
feat: add fuyu vision model and persimmon language model support
|
vor 6 Monaten |
|
AlpinDale
|
1efd0f89b7
feat: support FP8 for DeepSeekV2 MoE
|
vor 6 Monaten |
|
AlpinDale
|
bf15e1b4e8
chore: deprecation warning for beam search
|
vor 6 Monaten |
|
AlpinDale
|
7e9d4f3c71
chore: some more marlin cleanups
|
vor 6 Monaten |
|
AlpinDale
|
34fc26c869
chore: bump lmfe version to 0.10.3
|
vor 6 Monaten |
|
AlpinDale
|
23408b9b2b
chore: skip the driver worker
|
vor 6 Monaten |
|
AlpinDale
|
d8f9f0ec16
fix: prefix prefill kernels for fp32 data type
|
vor 6 Monaten |
|
AlpinDale
|
92bcbdf975
fix: megacore setting for TPU v5e-litepod
|
vor 6 Monaten |
|
AlpinDale
|
0c17c2a8a7
chore: add commit hash, clean up engine logs
|
vor 6 Monaten |
|
AlpinDale
|
cdc0e498a9
fix: illegal memory access in FP8 MoE kernel
|
vor 6 Monaten |
|
AlpinDale
|
b1e61268a8
bump torch to 2.3.1
|
vor 6 Monaten |
|
AlpinDale
|
05e45aeb53
fix: dtype mismatch for paligemma
|
vor 6 Monaten |
|
AlpinDale
|
500f3b654f
fix: support bias term in compressed-tensors quant
|
vor 6 Monaten |
|
AlpinDale
|
d2f38f6f81
chore: remove separate bias add
|
vor 6 Monaten |
|
AlpinDale
|
ddb28a80a3
fix: bump torch for rocm, unify CUDA_VISIBLE_DEVICES for cuda and rocm
|
vor 6 Monaten |
|
AlpinDale
|
a2d476183f
fix: remove scipy and re-implement CSR matrix
|
vor 6 Monaten |
|
AlpinDale
|
5ac65d2d49
chore: bump optimum-intel
|
vor 6 Monaten |
|
AlpinDale
|
cc6399792f
fix: keep consistent with how pytorch finds libcudart.so
|
vor 6 Monaten |
|
AlpinDale
|
63becc67c0
fix: prompt logprob detokenization
|
vor 6 Monaten |
|
AlpinDale
|
0ab35652d3
fix: llava 1.6 feature size calculation
|
vor 6 Monaten |
|
AlpinDale
|
058e629f8e
chore: refactor marlin python utils
|
vor 6 Monaten |
|
AlpinDale
|
c0c2b1ac20
fix: get_and_reset only when scheduler outputs are not empty
|
vor 6 Monaten |
|
AlpinDale
|
b9268be8e8
fix: engine timeout due to request abort
|
vor 6 Monaten |
|
AlpinDale
|
8a44866e00
restrict outlines to < 0.1
|
vor 6 Monaten |
|
AlpinDale
|
4501ae5f15
fix: neuron executor for adapters
|
vor 6 Monaten |
|
AlpinDale
|
16dff9babc
chore: enable bonus token in spec decoding for KV cache based models
|
vor 6 Monaten |
|
AlpinDale
|
4150b1ea3a
fix: adapter methods for OpenVINO executor
|
vor 6 Monaten |
|
AlpinDale
|
db73f03cdc
fix: use ParallelLMHead for MLPSpeculator
|
vor 6 Monaten |
|
AlpinDale
|
9622c59f8f
chore: support 2D input shape in MoE layer
|
vor 6 Monaten |
|
AlpinDale
|
4628caeae6
fix: missed these adapter methods for TPU executor
|
vor 6 Monaten |
