david/aphrodite-engine: PygmalionAI's large-scale inference engine pygmalion.chat It is designed to serve as the inference endpoint for the PygmalionAI website, and to allow serving the Pygmalion models to a large number of users with blazing fast speeds (thanks to vLLM's Paged Attention).

AlpinDale 7b6501bd05 tests: refactor model tests (#1078)		1 dag geleden
..
async_aphrodite	055c8905a3 api: add sampling/engine option to return only deltas or final output (#1035)	1 week geleden
basic_correctness	7b6501bd05 tests: refactor model tests (#1078)	1 dag geleden
benchmarks	9a7d5514c4 feat: introduce MQAphroditeEngine (#1056)	1 week geleden
compile	239a8cae25 torch.compile: register all-reduce operations as custom ops (#1050)	1 week geleden
core	f7f3fed265 feat: add async postprocessor (#925)	2 weken geleden
distributed	7b6501bd05 tests: refactor model tests (#1078)	1 dag geleden
encoder_decoder	a985143768 core: add cuda graph support for encoder-decoder models (#1051)	1 week geleden
endpoints	6212072245 api: support LoRA lineage and base model metadata management (#1072)	4 dagen geleden
engine	3bb0f07461 chore: rename `task_handler` to `worker` (#985)	2 weken geleden
kernels	7b6501bd05 tests: refactor model tests (#1078)	1 dag geleden
lora	3bb0f07461 chore: rename `task_handler` to `worker` (#985)	2 weken geleden
metrics	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
modeling	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
models	7b6501bd05 tests: refactor model tests (#1078)	1 dag geleden
multi_step	58aff3771d core: support prompt logprobs in multi-step (#1060)	6 dagen geleden
multimodal	2aabf8fcf7 vlm: fix errors on ragged NestedTensors (#953)	2 weken geleden
plugins	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
prefix_caching	6212072245 api: support LoRA lineage and base model metadata management (#1072)	4 dagen geleden
prompt_adapter	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
prompts	e1f3fd1e02 fix: test units (#201)	11 maanden geleden
quantization	6bdff60aab quant: support pre-quanted bitsandbytes checkpoints (#961)	2 weken geleden
samplers	2150bb5019 sampler: add range parameter for DRY (#855)	1 maand geleden
spec_decode	0859dc3bc0 tests: refactor speculative decoding tests to remove the async engine (#1021)	1 week geleden
tensorizer_loader	673621a3d2 xpu: refactor the model runner for tensor parallelism (#910)	3 weken geleden
tokenization	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
tool_use	0191c5efd1 tools: fix tool calls to more strictly follow OpenAI format (#1003)	1 week geleden
tpu	ea59784f59 tpu: remove torch._dynamo.reset() (#952)	2 weken geleden
weight_loading	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
worker	314fa7f7d9 fix: encoder-decoder models for beam search (#1064)	5 dagen geleden
__init__.py	2755a48d51 merge dev branch into main (#153)	1 jaar geleden
conftest.py	7b6501bd05 tests: refactor model tests (#1078)	1 dag geleden
test_cache_block_hashing.py	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
test_config.py	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
test_embedded_commit.py	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
test_inputs.py	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
test_logits_processor.py	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
test_regression.py	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
test_sampling_params.py	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
test_scalartype.py	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
test_sequence.py	0dfa6b60ec core: support logprobs with multi-step scheduling (#963)	2 weken geleden
test_sharded_state_loader.py	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
test_utils.py	c6c91edab7 ci: update & overhaul test units (#769)	1 maand geleden
utils.py	7b6501bd05 tests: refactor model tests (#1078)	1 dag geleden