Assertions reference
Outcome, behaviour, change, exposure and coverage assertions, every definition type, severity, results and how an Event is derived from them.
v1Updated
Assertions are the contract between what you meant and what a run observed. They are evaluated deterministically by a versioned evaluator (currently 1.0.0); the version is recorded on every result so you can always see which rule, at which version, produced a conclusion.
Shape
Each assertion in a Watch definition has five fields:
| Field | Type | Meaning |
|---|---|---|
id |
string, up to 64 chars | Unique within the Watch |
kind |
outcome, behaviour, change, exposure, coverage |
Which question it answers (below) |
severity |
critical, high, medium, low, info |
Weight of a failure; the Event takes the highest failing severity |
description |
string, 1 to 500 chars | Shown in the app and in Event summaries |
definition |
object discriminated on type |
What is checked |
{
"id": "api-status",
"kind": "outcome",
"severity": "high",
"description": "Quote API responds 200",
"definition": { "type": "httpStatus", "stepId": "post-quote", "status": 200 }
}Kinds
| Kind | Decides | Typical definitions |
|---|---|---|
outcome |
whether customers got through | allStepsPass, stepOutcome, httpStatus, jsonPath, contentContains, latencyBelow |
behaviour |
material change while outcome may still pass | behaviourStable |
change |
recorded difference between observations | dnsAnswerContains, tlsValidDays |
exposure |
externally observable exposure fact | exposureNone |
coverage |
whether the Watch produced enough evidence | coverageLocations |
Outcome and behaviour are always separate signals. Failing outcome assertions drive the materiality policy and open outcome Events. behaviour results move a Journey to Changed and only open an Event when the Watch’s behaviourOpensEvent policy allows it. coverage results explain Unknown rather than failure. The kind is a label you choose; any definition type may be used under any kind, and the table above shows the pairing the composer uses.
Definition types
Every stepId must name a step in the same Watch; the API rejects a definition that references an unknown step.
type |
Fields | Passes when | Reads |
|---|---|---|---|
allStepsPass |
none | No step in the Observation failed or errored | step statuses |
stepOutcome |
stepId |
The step passed (not_applicable if it was skipped) |
step status |
latencyBelow |
ms, optional stepId |
The step’s durationMs, or the whole probe’s when stepId is omitted, is below ms |
step or probe timing |
httpStatus |
stepId, status or range: [min, max] (one required) |
The recorded status matches | HTTP status of the step |
contentContains |
stepId, text |
The captured body or page text contains text |
body excerpt or page text |
jsonPath |
stepId, path, equals |
The value at path (for example $.data.items[0].id) deep-equals equals |
JSON body of the step |
dnsAnswerContains |
stepId, value |
The DNS answers include value |
dns.resolve answers |
tlsValidDays |
stepId, days |
The certificate’s notAfter is at least days after the Observation ended |
tls.inspect summary |
behaviourStable |
tolerance: strict, normal, loose |
The behaviour fingerprint is within tolerance of the baseline | fingerprint and baseline |
coverageLocations |
min |
At least min distinct locations observed the slot |
location count |
exposureNone |
none | The Observation recorded zero exposures | metrics.exposures |
All of the following validate.
[
{ "type": "allStepsPass" },
{ "type": "stepOutcome", "stepId": "submit" },
{ "type": "latencyBelow", "stepId": "wait-price", "ms": 10000 },
{ "type": "latencyBelow", "ms": 30000 },
{ "type": "httpStatus", "stepId": "post-quote", "status": 200 },
{ "type": "httpStatus", "stepId": "post-quote", "range": [200, 299] },
{ "type": "contentContains", "stepId": "confirm", "text": "Order number" },
{ "type": "jsonPath", "stepId": "post-quote", "path": "$.premium.currency", "equals": "GBP" },
{ "type": "dnsAnswerContains", "stepId": "apex-a", "value": "203.0.113.10" },
{ "type": "tlsValidDays", "stepId": "cert", "days": 14 },
{ "type": "behaviourStable", "tolerance": "normal" },
{ "type": "coverageLocations", "min": 2 },
{ "type": "exposureNone" }
]Outcome assertions
[
{ "id": "quote-all", "kind": "outcome", "severity": "critical", "description": "Quote journey completes", "definition": { "type": "allStepsPass" } },
{ "id": "order-number", "kind": "outcome", "severity": "critical", "description": "Confirmation shows an order number", "definition": { "type": "contentContains", "stepId": "confirm", "text": "Order number" } },
{ "id": "premium-currency", "kind": "outcome", "severity": "high", "description": "Premium is quoted in GBP", "definition": { "type": "jsonPath", "stepId": "post-quote", "path": "$.premium.currency", "equals": "GBP" } },
{ "id": "fast-enough", "kind": "outcome", "severity": "medium", "description": "Quote API answers within 3 s", "definition": { "type": "latencyBelow", "stepId": "post-quote", "ms": 3000 } }
]Latency breaches are usually given medium severity so they degrade the Journey rather than mark it critical. The failure boundary of an Event is the first step that failed or errored; assertion results explain why.
Behaviour assertions
{ "id": "quote-behaviour", "kind": "behaviour", "severity": "info", "description": "Quote form behaviour stable", "definition": { "type": "behaviourStable", "tolerance": "normal" } }behaviourStable compares the Observation’s behaviour fingerprint with the accepted baseline for the active Watch version. The fingerprint has six dimensions, each producing a normalised difference from 0 (identical) to 1 (unrelated): step timings, DOM structure, screenshot perceptual hash, response shape, redirect chain and third-party hosts. The assertion fails when any dimension exceeds the threshold for the chosen tolerance:
| Dimension | strict |
normal |
loose |
|---|---|---|---|
| Timing (ratio to baseline) | 0.5 | 1.0 | 2.0 |
| DOM structure | 0 | 0.34 | 0.34 |
| Screenshot | 0 | 0.34 | 0.5 |
| Response shape | 0 | 0 | 0.5 |
| Redirect chain | 0 | 0 | 0.5 |
| Third-party hosts | 0 | 0.34 | 0.67 |
normal tolerates a third of the DOM or screenshot changing but flags any change in response shape or redirect chain. Without a fingerprint on the Observation, or without a baseline yet, the result is not_applicable. Accepting a change in the app updates the baseline and records who accepted it. Semantic materiality (a model’s opinion of whether a change matters) annotates a behaviour Event with provenance; it never decides a result.
Change and exposure assertions
[
{ "id": "apex-points-at-edge", "kind": "change", "severity": "high", "description": "Apex A record points at the edge", "definition": { "type": "dnsAnswerContains", "stepId": "apex-a", "value": "203.0.113.10" } },
{ "id": "cert-renewal", "kind": "change", "severity": "medium", "description": "Certificate valid for at least 14 days", "definition": { "type": "tlsValidDays", "stepId": "cert", "days": 14 } },
{ "id": "origin-hidden", "kind": "exposure", "severity": "high", "description": "No origin reachable without the edge", "definition": { "type": "exposureNone" } }
]exposureNone reads the exposures metric an origin_bypass probe records; it is not_applicable on probes that do not record one.
Coverage assertions
{ "id": "two-locations", "kind": "coverage", "severity": "low", "description": "At least two locations report each slot", "definition": { "type": "coverageLocations", "min": 2 } }coverageLocations is evaluated against the number of distinct locations that observed the same schedule slot (provider:networkClass:region keys). Pair it with locationPolicy.minLocations on the Watch. When coverage cannot be met, for example two of three regions returned unknown because of browser capacity, the Journey shows Unknown with the reason and no critical Event opens.
Results
Each evaluation is stored as an AssertionResult:
| Field | Meaning |
|---|---|
assertionId |
The assertion’s id |
kind |
Copied from the assertion |
result |
pass, fail, not_applicable or unknown |
confidence |
0 to 1; 1 for every deterministic evaluator |
evaluatorVersion |
The evaluator that produced the result |
message |
Human-readable explanation, for example HTTP 502, expected 200 |
When the Observation’s outcome is unknown (runner error, timeout before first byte, browser capacity, blocked by bot protection, runner disconnected, secret unavailable, internal error) every assertion result is unknown with the reason in its message. An assertion that cannot be evaluated on an otherwise good run, such as httpStatus on a step that recorded no status, is also unknown; one whose subject was skipped or is not present is not_applicable. Neither counts as a target failure.
Severity and materiality
The Event opened for a failing run takes the highest severity among the failing assertions. The Watch’s materiality policy then decides whether an Event opens at all: by default two consecutive failing runs with the same failure boundary within 30 minutes, from at least one location, and automatic resolution after two consecutive passes. See materiality policy for the fields and POST /v1/watches/materiality-preview to see how many Events a policy would have opened over a past period.
Unknown is a state
A probe that could not run yields unknown with one of these reasons: runner_error, timeout_before_first_byte, browser_capacity, blocked_by_bot_protection, runner_disconnected, secret_unavailable, internal_error. Unknown never counts as a target failure, never counts against availability, and never opens an outcome Event. Three consecutive unknowns open a low-severity coverage Event so a persistent gap is visible.