PygmalionAI's large-scale inference engine
pygmalion.chat

It is designed to serve as the inference endpoint for the PygmalionAI website, and to allow serving the Pygmalion models to a large number of users with blazing fast speeds (thanks to vLLM's Paged Attention).

32 Commits

79 Ramas

36 Releases

AlpinDale d40a8d6bb0 chore: bind single_query_cached_kv_attention to python		hace 1 año
aphrodite	b48fe85378 chore: utilities for modeling	hace 1 año
assets	fefbf029c9 revert previous commit	hace 1 año
kernels	d40a8d6bb0 chore: bind single_query_cached_kv_attention to python	hace 1 año
.gitignore	3c3944153c feat: add generic attention and FP32 dtype kernels	hace 1 año
LICENSE	fefbf029c9 revert previous commit	hace 1 año
README.md	fefbf029c9 revert previous commit	hace 1 año
requirements.txt	fefbf029c9 revert previous commit	hace 1 año

Aphrodite - The Pygmalion Backend

Work in Progress

Aphrodite is the backend service for PygmalionAI, built on top of FastChat, vLLM, SkyPilot, and more.

Currently a work in progress, not remotely functional.

README.md

Aphrodite - The Pygmalion Backend

Work in Progress