david/aphrodite-engine: PygmalionAI's large-scale inference engine pygmalion.chat It is designed to serve as the inference endpoint for the PygmalionAI website, and to allow serving the Pygmalion models to a large number of users with blazing fast speeds (thanks to vLLM's Paged Attention). @ 172dee25731746a9e8f916ba2d4a8c6b71c16f02

AlpinDale fa84f8102e kernels: split marlin kernels for faster compile, fix MoE, temporarily remove HQQ (#1119)		5 日前
..
__init__.py	a5bfc2bc3d VLM: add support for LLaVA-Onevision model (#1100)	2 週間前
arctic.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
baichuan.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
bart.py	a985143768 core: add cuda graph support for encoder-decoder models (#1051)	1 ヶ月前
blip.py	7ce3174039 VLM: refactor blip models to support composite weight loading (#1098)	2 週間前
blip2.py	7ce3174039 VLM: refactor blip models to support composite weight loading (#1098)	2 週間前
bloom.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
chameleon.py	651678d2df VLM: use `SequenceData.from_token_counts` to create dummy data (#1093)	2 週間前
chatglm.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
clip.py	80be38ca6f chore: expose phi3_v num_crops as an mm_processor_kwargs (#1117)	1 週間前
commandr.py	135dfd648b fix: LoRA support for Cohere and Jamba models (#1004)	1 ヶ月前
dbrx.py	b65449b5ad moe: refactor DBRX experts to support FusedMoE (#1095)	2 週間前
decilm.py	9022c6d869 remove progress_bar imports	3 ヶ月前
deepseek.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
deepseek_v2.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
eagle.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
exaone.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
falcon.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
fuyu.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
gemma.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
gemma2.py	b33cf04386 quants: add bitsandbytes support for gemma2 model (#1026)	1 ヶ月前
gpt2.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
gpt_bigcode.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
gpt_j.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
gpt_neox.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
granite.py	11f49b5341 fix: granite logit scale in logit computation (#1054)	1 ヶ月前
idefics2_vision_model.py	f1d0b77c92 [0.6.0] Release Candidate (#481)	5 ヶ月前
interfaces.py	0b8b407b6d feat: support profiling with multiple multi-modal inputs per prompt (#712)	5 ヶ月前
intern_vit.py	f56d6b396a vlm: fallback to SDPA for ViT models on CPU backend (#982)	1 ヶ月前
internlm2.py	7632f91429 fix: InternLM2 model with Tensor Parallel (#980)	1 ヶ月前
internvl.py	91d03c04d2 VLM: refactor composite weight loading logic (#1097)	2 週間前
jais.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
jamba.py	135dfd648b fix: LoRA support for Cohere and Jamba models (#1004)	1 ヶ月前
llama.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
llama_embedding.py	9022c6d869 remove progress_bar imports	3 ヶ月前
llava.py	91d03c04d2 VLM: refactor composite weight loading logic (#1097)	2 週間前
llava_next.py	7ce3174039 VLM: refactor blip models to support composite weight loading (#1098)	2 週間前
llava_next_video.py	91d03c04d2 VLM: refactor composite weight loading logic (#1097)	2 週間前
llava_onevision.py	a5bfc2bc3d VLM: add support for LLaVA-Onevision model (#1100)	2 週間前
mamba.py	3bb0f07461 chore: rename `task_handler` to `worker` (#985)	1 ヶ月前
mamba_cache.py	a113309876 kernel: add meta functions for ops to prevent graph breaks (#1019)	1 ヶ月前
medusa.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
minicpm.py	ce7b602f03 model: add support for MiniCPM-3 (#1044)	1 ヶ月前
minicpm3.py	ce7b602f03 model: add support for MiniCPM-3 (#1044)	1 ヶ月前
minicpmv.py	651678d2df VLM: use `SequenceData.from_token_counts` to create dummy data (#1093)	2 週間前
mixtral.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
mixtral_quant.py	fa84f8102e kernels: split marlin kernels for faster compile, fix MoE, temporarily remove HQQ (#1119)	5 日前
mlp_speculator.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
molmo.py	acc0c727c8 vlm: add support for molmo vision model (#1069)	1 ヶ月前
mpt.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
na_vit.py	9f3e7c86e2 feat: add fused Marlin MoE kernel (#934)	1 ヶ月前
nemotron.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
olmo.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
olmoe.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
opt.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
orion.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
paligemma.py	91d03c04d2 VLM: refactor composite weight loading logic (#1097)	2 週間前
persimmon.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
phi.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
phi3.py	ec17b6c4d0 fix: Phi3.5 Mini and MoE LoRA inference (#1070)	1 ヶ月前
phi3_small.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
phi3v.py	80be38ca6f chore: expose phi3_v num_crops as an mm_processor_kwargs (#1117)	1 週間前
phimoe.py	ec17b6c4d0 fix: Phi3.5 Mini and MoE LoRA inference (#1070)	1 ヶ月前
pixtral.py	651678d2df VLM: use `SequenceData.from_token_counts` to create dummy data (#1093)	2 週間前
qwen.py	651678d2df VLM: use `SequenceData.from_token_counts` to create dummy data (#1093)	2 週間前
qwen2.py	c90abcc603 VLM: add pipeline parallelism support for Qwen2-VL (#1103)	2 週間前
qwen2_moe.py	5224389dae chore: skip loading extra bias for qwen2 moe GPTQ (#1011)	1 ヶ月前
qwen2_vl.py	c90abcc603 VLM: add pipeline parallelism support for Qwen2-VL (#1103)	2 週間前
siglip.py	a5bfc2bc3d VLM: add support for LLaVA-Onevision model (#1100)	2 週間前
solar.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
stablelm.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
starcoder2.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前
ultravox.py	18e0b0e932 chore: support loading weights by ID within models (#1116)	1 週間前
utils.py	91d03c04d2 VLM: refactor composite weight loading logic (#1097)	2 週間前
xverse.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	1 ヶ月前