Define an SLO for tool calls

Define the indicator

Choose a user-relevant tool operation and an observation window. Specify the denominator of eligible calls and the condition for a good call. Google SRE recommends grounding service objectives in meaningful indicators rather than collecting arbitrary metrics.

Illustrative objective

For a read tool, define good as “returns a valid response within 2 seconds” and eligible as all admitted reads during a rolling 28-day window. A target of 99 percent is an example policy, not a measured result or a suitable target for every service. Decide explicitly how timeouts, rejected calls and planned maintenance are treated.

Agent-specific distinction

A fast HTTP 200 can still contain an irrelevant answer. Track protocol reliability separately from evaluation of task correctness. Do not combine them into an unexplained single score.

Acceptance and limits

Construct a ten-call fixture with one timeout and confirm the ratio follows the declared denominator. Check that the dashboard and alert use the same definitions. Set a response policy for budget consumption, such as pausing risky changes. An SLO is a management objective, not a guarantee to an individual caller or proof of semantic accuracy.