Branch:
feat/gumbell-softmax-experiment
0.6.4
aarch64-docker
astarte
attentionstate
banned_strings
bitnet
bnb_tp
cache_hit_rate
cfg
cfg_take2
cfg_take3
chore/test-updates
chunked-caching
chunked_caching_fix
configure
control-vectors
cuda_lora
custom_op_check
deepseek_v3
dependabot/npm_and_yarn/docs/cross-spawn-7.0.5
dev
draft-default-len
dry
dry-concurrency
dry-fix
dry-sampler
dry-seq-breakers-pad
dry_perf_improvement
dry_range
dual_chunk_attn
dynatempagain
eetq_kernels
exl2_return
feat/function-calling
feat/grok-1
feat/gumbell-softmax-experiment
feat/jamba-support
feat/lora-loader-enhancements
feat/perplexity
feat/rust
feat/t5-support
feat/tree-attention
feat/typ_p_threshold
fix/mixtral-gguf
fix_sampler_test
flute
fp8-quant
fully-sharded-lora-fix
geppetto
get_last_latency_hack
hunyuan
improved_dry
int4_weights
jinja2
linear_weights_light
llama3-rope
lm_head_lora
lora-scaling
machete
main
mistral_common_1.5.0
mistral_skip_special_tokens
mqaphrodite
multi-step
multinode_dynatemp
mypy-round-1
nf_quant
no_dry_seqs
no_repeat_ngram
outlines-import
premature-exit-async
profile_no_compile
prompt_logprobs_mem
punica_xpu
pyinstaller
quip-return
rc_054
rejection_sampling_kernels
rep_pen_range
revert_871
sampler_order_string
sampler_order_v2
sampler_priorty
sampler_refactor
sampler_tests
sampling-kernels
sampling_experiments
sampling_kernels
seqdata-factor
shrek
spmd-optimization
spmd_optim
swinv2
temp_last_warning
tools-api
top-nsigma
ultravox
uneven-head-size-flashinfer
update_benchmarks
v0.6.4.post1
vectorized_dry
video
windows_support
xqa
v0.6.6
v0.6.5
v0.6.4.post1
v0.6.4
v0.6.3.post1
v0.6.3
v0.6.2.post1
v0.6.2
v0.6.1.post1
v0.6.1
v0.6.0.post1
v0.6.0
rc_054
v0.5.3
v0.5.2
v0.5.1
v0.5.0
v0.4.9
v0.4.8
v0.4.7
v0.4.6
v0.4.5
v0.4.4
v0.4.3
v0.4.2
v0.4.1
v0.4
v0.3.7
v0.3.6
v0.3.5
v0.3.4
v0.3.3
v0.3.2
v0.3.1
v0.3
AlpinDale
15a0454172
feat: FP8 KV Cache (#185)
|
1 year ago | |
|---|---|---|
| .. | ||
| quant_utils.cuh | 15a0454172 feat: FP8 KV Cache (#185) | 1 year ago |