Documentation
MLSrvyn Documentation
Connect MLSrvyn to your inference fleet in under 30 minutes. Full API reference, configuration schema, and architectural context included.
Quickstart Guide
Connect your first model endpoint and see MLSrvyn detecting traffic patterns in under 30 minutes.
Get startedConfiguration Reference
Full schema for mlsrvyn.yaml: archetypes, SLO targets, scaling bounds, bin-packing rules, and alerting hooks.
Explore configAPI Reference
REST API endpoints and Python SDK for programmatic fleet management, policy updates, and metric queries.
View APIArchitecture Overview
How MLSrvyn integrates with your existing GPU orchestration layer — Kubernetes, Slurm, or bare-metal.
Read architectureHow MLSrvyn connects to your fleet
MLSrvyn is a sidecar-style control plane that sits between your traffic source and your inference replicas. It does not handle request traffic directly — it adjusts replica counts and placement in response to observed latency and utilization metrics.
(Envoy / Nginx / ALB)
(GPU nodes)
(scales replica pool)
The control plane receives a telemetry stream (latency histograms, GPU utilization, queue depth) from your inference gateway and issues scaling decisions to your orchestration layer. No traffic passes through MLSrvyn — only signals and commands.