Configuration Reference

Complete schema for mlsrvyn.yaml. Each model endpoint gets its own configuration block. Workspace-level settings apply globally.

Top-level structure

mlsrvyn.yaml

workspace: string         # required
orchestrator: kubernetes | slurm | bare_metal
models:                   # list of model configs
  - name: string
    endpoint: url
    archetype: bursty | steady | event_driven | periodic | auto
    slo: {...}
    scaling: {...}
    bin_packing: {...}    # optional
    alerts: {...}          # optional

SLO block

Field	Type	Description
`p50_ms`	integer	Median latency target in milliseconds. Optional — used as soft scaling threshold.
`p90_ms`	integer	90th percentile latency target. Optional.
`p99_ms`	integer	99th percentile latency target. Required for SLO enforcement.
`ttft_p90_ms`	integer	Time-to-first-token p90 target. LLM endpoints only.

Scaling block

Field	Type	Default	Description
`min_replicas`	integer	1	Minimum replicas to keep warm at all times.
`max_replicas`	integer	20	Maximum replicas allowed.
`scale_up_slo_pct`	integer	80	Scale out when p99 reaches this percentage of the SLO budget.
`scale_batch_size`	integer	1	Number of replicas to add per scale-out event.
`prewarm_minutes_before_peak`	integer	—	Bursty archetype: minutes before predicted peak to pre-warm replicas.
`sessions_per_replica`	integer	—	Steady (LLM) archetype: max concurrent sessions per replica before scaling.
`scaling_metric`	string	p99_latency	Override: `p99_latency`, `concurrent_sessions`, `queue_depth`, `gpu_utilization`.

Bin-packing block

mlsrvyn.yaml — bin_packing section

bin_packing:
  enabled: true
  gpu_memory_reserve_mb: 2048   # keep 2GB free on each GPU
  co_locate_with:               # models allowed to share GPU
    - embedding-svc
    - rerank-v2
  isolation_required: false     # set true for regulated workloads

Next steps

API Reference — manage config programmatically
Recommendation Systems — full bursty example
LLM Serving — full steady example