The example deploys the 82-million-parameter Kokoro-82M model in a custom container on a SageMaker AI real-time endpoint, with deployment handled through AWS CDK stacks.
The guide bases its policy on two measurements from a live endpoint: Q_max, the highest concurrency a single instance can sustain without breaching a latency SLO, and T_total, the end-to-end scale-out time. AWS says these values can be used to calculate scale-out and scale-in thresholds and cooldowns. The endpoint scales on SageMaker AI's native ConcurrentRequestsPerModel metric.
AWS notes that the metric reports a Maximum over a 10-second window, while the load client measures mean in-flight requests. Its tooling measures the relationship between the two at each concurrency level; in the reported Kokoro-82M run, the ratio was 1.0. The source does not establish that these thresholds, ratios, costs, or latency results will transfer to other models, traffic patterns, instance types, or SLOs.