ACORN GLOBAL · ENGINEERING FIELD GUIDE · PERFORMANCE ENGINEERING
Measure and improve AI workflow latency
Latency includes model inference, network transport, tool handoffs and retries. A single fast response says little about p95 performance.
Mesurer la latence IA — mesures vérifiables, aucune garantie ni résultat client inventé.
Look for these observable signals
- p95 latency exceeds a stated service objective
- Tool chains accumulate queue and network delays
- Long-tail timeouts are hidden by average measurements
A REPRODUCIBLE METHOD
Four steps to a useful baseline
Define a user-visible start and finish and set the target latency.
Collect an authorized time series with sample size and measurement interval.
Separate the model, tool, queue and transport spans.
Compare equivalent workloads before and after each bounded change.
Evidence worth retaining
p50 and p95 definitions, baseline, workload, sample count, timing source and acceptance criteria.
Limit: Do not compare two models using different workloads or infer improvements without repeated observations.
If required measurements are unavailable, record them as not measured. Never replace an unknown value with zero or claim a successful repair without a verified readback.