Xinference

Serve LLMs, embeddings, and media models from one Apache-2.0 platform.

Install Repository Compare

Project.

License: Apache 2.0
Platforms: Linux, Macos, Windows, Docker
Docker: Yes
Commercial use: Allowed
Stars: 9.6k
Language: Python
Last commit: Sep 11, 2026
Tags: servingschedulingopenai-api

One-box alternative to juggling Ollama plus separate embedding servers.

About.

Xinference launches, schedules, and serves many models from one platform: LLMs, embeddings, rerankers, image and audio models behind an OpenAI-shaped API with a built-in chat UI. Apache-2.0 and very active.

multi-model serving · built-in UI · GPU scheduling · OpenAI-compatible API

Alternatives.

All LLM infrastructure

OOllamaLocal model runner with a one-line CLI and a REST API.Compare
Lllama.cppC/C++ runtime that made local GGUF models practical.Compare
LLocalAIOpenAI-compatible API that can sit in front of several local backends.Compare
LllamafileSingle-file LLM runtime from Mozilla. Download, chmod, run.Compare

Project.

About.

Alternatives.

Related projects.

Project.

About.

Alternatives.

Related projects.