david/aphrodite-engine: PygmalionAI's large-scale inference engine pygmalion.chat It is designed to serve as the inference endpoint for the PygmalionAI website, and to allow serving the Pygmalion models to a large number of users with blazing fast speeds (thanks to vLLM's Paged Attention).

AlpinDale 766ea79b89 vlm: fix feature size calculation for llava-next models (#1079)		3 giorni fa
..
__init__.py	ec17b6c4d0 fix: Phi3.5 Mini and MoE LoRA inference (#1070)	6 giorni fa
arctic.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
baichuan.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
bart.py	a985143768 core: add cuda graph support for encoder-decoder models (#1051)	1 settimana fa
blip.py	f56d6b396a vlm: fallback to SDPA for ViT models on CPU backend (#982)	2 settimane fa
blip2.py	b4a1e2fd02 vlm: add tensor parallel support for vision transformer models (#971)	2 settimane fa
bloom.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
chameleon.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
chatglm.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
clip.py	4d14bd1fe5 vlm: add multi-input support for LLaVA and InternVL models (#1002)	2 settimane fa
commandr.py	135dfd648b fix: LoRA support for Cohere and Jamba models (#1004)	2 settimane fa
dbrx.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
decilm.py	9022c6d869 remove progress_bar imports	2 mesi fa
deepseek.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
deepseek_v2.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
eagle.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
exaone.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
falcon.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
fuyu.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
gemma.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
gemma2.py	b33cf04386 quants: add bitsandbytes support for gemma2 model (#1026)	1 settimana fa
gpt2.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
gpt_bigcode.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
gpt_j.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
gpt_neox.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
granite.py	11f49b5341 fix: granite logit scale in logit computation (#1054)	1 settimana fa
idefics2_vision_model.py	f1d0b77c92 [0.6.0] Release Candidate (#481)	4 mesi fa
interfaces.py	0b8b407b6d feat: support profiling with multiple multi-modal inputs per prompt (#712)	3 mesi fa
intern_vit.py	f56d6b396a vlm: fallback to SDPA for ViT models on CPU backend (#982)	2 settimane fa
internlm2.py	7632f91429 fix: InternLM2 model with Tensor Parallel (#980)	2 settimane fa
internvl.py	41ceb754a6 vlm: fix internvl2 inference with various num_patches (#1030)	1 settimana fa
jais.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
jamba.py	135dfd648b fix: LoRA support for Cohere and Jamba models (#1004)	2 settimane fa
llama.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
llama_embedding.py	9022c6d869 remove progress_bar imports	2 mesi fa
llava.py	4d14bd1fe5 vlm: add multi-input support for LLaVA and InternVL models (#1002)	2 settimane fa
llava_next.py	766ea79b89 vlm: fix feature size calculation for llava-next models (#1079)	3 giorni fa
llava_next_video.py	be59e30139 vlm: add support for video modality + llava next video (#1014)	2 settimane fa
mamba.py	3bb0f07461 chore: rename `task_handler` to `worker` (#985)	2 settimane fa
mamba_cache.py	a113309876 kernel: add meta functions for ops to prevent graph breaks (#1019)	1 settimana fa
medusa.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
minicpm.py	ce7b602f03 model: add support for MiniCPM-3 (#1044)	1 settimana fa
minicpm3.py	ce7b602f03 model: add support for MiniCPM-3 (#1044)	1 settimana fa
minicpmv.py	548e864404 models: add support for QwenVL (#995)	2 settimane fa
mixtral.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
mixtral_quant.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
mlp_speculator.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
molmo.py	acc0c727c8 vlm: add support for molmo vision model (#1069)	6 giorni fa
mpt.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
na_vit.py	9f3e7c86e2 feat: add fused Marlin MoE kernel (#934)	2 settimane fa
nemotron.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
olmo.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
olmoe.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
opt.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
orion.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
paligemma.py	46d577f019 vlm: fix siglip layernorm and paligemma weight loading (#991)	2 settimane fa
persimmon.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
phi.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
phi3.py	ec17b6c4d0 fix: Phi3.5 Mini and MoE LoRA inference (#1070)	6 giorni fa
phi3_small.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
phi3v.py	4d14bd1fe5 vlm: add multi-input support for LLaVA and InternVL models (#1002)	2 settimane fa
phimoe.py	ec17b6c4d0 fix: Phi3.5 Mini and MoE LoRA inference (#1070)	6 giorni fa
pixtral.py	1721bea53a vlm: add support for Pixtral model (#1022)	1 settimana fa
qwen.py	6212072245 api: support LoRA lineage and base model metadata management (#1072)	6 giorni fa
qwen2.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
qwen2_moe.py	5224389dae chore: skip loading extra bias for qwen2 moe GPTQ (#1011)	2 settimane fa
qwen2_vl.py	6212072245 api: support LoRA lineage and base model metadata management (#1072)	6 giorni fa
siglip.py	4d14bd1fe5 vlm: add multi-input support for LLaVA and InternVL models (#1002)	2 settimane fa
solar.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
stablelm.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
starcoder2.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa
ultravox.py	8eb4a3cfd3 vlm: support multiple audios per prompt for Ultravox (#990)	2 settimane fa
utils.py	a8bdd488b9 distributed: support pipeline parallelism for internvl and internlm2 (#965)	2 settimane fa
xverse.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 settimane fa