Diagnose webhooks

Find why a webhook failed with bounded searches, stable IDs, event detail, the delivery retry chain, and the DLQ.

Start broad, then narrow by stable IDs. The goal is to gather enough evidence for an audit trail while making as few queries as possible.

flowchart TD
    H["health<br/>healthy enough to continue?"] --> S["events search<br/>find candidates in bounds"]
    S --> G["events get / diff<br/>inspect one event"]
    G --> C["forwards chain<br/>per-attempt timeline"]
    C --> D{Terminal & replayable?}
    D -->|yes| R["replay (remediate)"]
    D -->|validating a fix| W["expect / tail"]
    H -.->|dlq_depth > 0| Q["dlq list<br/>triage dead letters"]
    Q --> D

Start with health

When the question is "is this forwarder healthy enough to continue?", begin with health.

repost health \
  --bucket stripe-prod \
  --forwarder prod-api \
  --window 30m \
  --fail-on 'success_rate < 0.99 || dlq_depth > 0' \
  --json

{
  "schema": "repost.health/v1",
  "data": {
    "bucket": { "id": "bkt_01HV9S", "slug": "stripe-prod", "name": "Stripe production" },
    "window": "30m",
    "since": "2026-06-19T08:00:00Z",
    "until": "2026-06-19T08:30:00Z",
    "forwarders": [
      {
        "forwarder_id": "fwd_01HV9S",
        "forwarder_name": "prod-api",
        "success_rate": 0.9722,
        "failed": 4,
        "pending_retries": 2,
        "dlq_depth": 1,
        "p95_latency_ms": 1840.25,
        "last_delivery_at": "2026-06-19T08:29:51Z"
      }
    ]
  }
}

--fail-on turns a threshold into a process failure (partial_failure, exit 8) so CI and agents do not parse text. It accepts success_rate and dlq_depth with comparisons and ||. Here failed, pending_retries, and dlq_depth are non-zero, so continue into inspection.

Search events within bounds

Search only the bucket and time range the workflow needs.

repost events search 'method:POST AND path:/stripe/*' \
  --bucket stripe-prod \
  --since 30m \
  --limit 20 \
  --json

Page older results with --cursor. It is forward-only and bound to this exact query and window. Don't widen --limit to dodge paging; narrow the query. Full query grammar and cursor rules are in Search & filters.

Inspect an event

Move from search results to a single event by ID.

repost events get evt_01JZ8V1 --json
repost events diff evt_01JZ8V1 evt_01JZ8UZ --json

{
  "schema": "repost.events.get/v1",
  "data": {
    "event": {
      "id": "evt_01JZ8V1",
      "bucket_id": "bkt_01HV9S",
      "method": "POST",
      "path": "/stripe/webhook",
      "received_at": "2026-06-19T08:31:22Z",
      "response_status": 500,
      "request": {
        "headers": {
          "stripe-signature": ["t=1718061000,v1=[redacted:hmac:9f2a1c4d8b07]"],
          "content-type": ["application/json"]
        },
        "signature_header_present": true,
        "body": { "truncated": false, "bytes_total": 1284, "bytes_shown": 1284, "sha256": "6b8d4f0b…", "content": "{\"type\":\"invoice.payment_succeeded\"}" },
        "body_json": { "type": "invoice.payment_succeeded", "data": { "object": { "id": "in_123" } } }
      },
      "response": {
        "status": 500,
        "headers": { "content-type": ["text/plain"] },
        "body": { "truncated": false, "bytes_total": 36, "bytes_shown": 36, "sha256": "91c6f4b2…", "content": "database connection timeout" }
      }
    },
    "redactions": ["request.headers.stripe-signature"]
  }
}

events get truncates the body by default. Always check body.truncated; if it is true, re-fetch with --full for the complete payload. Use events diff to explain why one webhook behaved differently from another.

Credential headers and secret-like fields are redacted; PII is not. See Transcript safety before copying any payload into a transcript.

Inspect delivery attempts

Find failed deliveries with --failed, then pull the retry chain for one event and forwarder.

repost forwards search --bucket stripe-prod --forwarder prod-api --since 30m --failed --json
 
repost forwards chain evt_01JZ8V1 --bucket stripe-prod --forwarder prod-api --json

{
  "schema": "repost.forwards.chain/v1",
  "data": {
    "event_id": "evt_01JZ8V1",
    "forwarder_id": "fwd_01HV9S",
    "attempts": [
      { "forward_id": "fwd_attempt_01JZ8UZ", "attempt": 1, "type": "EXTERNAL", "status": "FAILED", "response_code": 503, "latency_ms": 220, "created_at": "2026-06-19T08:31:25Z", "started_at": "2026-06-19T08:31:25Z", "will_retry": false, "retried": true },
      { "forward_id": "fwd_attempt_01JZ8V2", "attempt": 3, "type": "EXTERNAL", "status": "FAILED", "response_code": 500, "latency_ms": 1840, "created_at": "2026-06-19T08:33:26Z", "started_at": "2026-06-19T08:34:10Z", "will_retry": true, "retried": false, "next_retry_at": "2026-06-19T08:35:10Z" }
    ]
  }
}

forwards search finds candidate attempts fast (use --failed rather than a status: query); forwards chain is the authoritative per-attempt timeline for one event and forwarder: order, response code, latency, retry intent, and terminal status. Search to locate, chain to drill in.

Read the four timing and retry fields precisely:

Field	Meaning
`created_at`	When the attempt was scheduled. For a retry this is set when the retry is queued, so it can precede the request by minutes. `--since` / `--until` bound this field, and both `forwards search` and the chain are ordered by it.
`started_at`	When the request actually left. Absent while an attempt is still scheduled; on `forwards search` it is derived from the completion stamp and the measured duration, so it is absent there for an attempt that has not finished.
`will_retry`	Retry intent: a retry is scheduled and has not run yet (`next_retry_at` is set). It turns `false` once that retry exists as its own attempt.
`retried`	Whether a later attempt actually continued this one. Use it to tell an attempt that was retried from one that ended the chain; `will_retry` alone cannot say that.

The chain's will_retry / next_retry_at / retried are the forwarder's automatic retries (its retry policy). Deliberately re-sending an event is a replay, a separate, manual action covered in Replay deliveries — a replay starts its own chain and never sets retried on the attempts it re-sends.

Triage the DLQ

When health reports DLQ depth, inspect the DLQ rows before replaying.

repost dlq list --bucket stripe-prod --forwarder prod-api --limit 20 --json

{
  "schema": "repost.dlq.list/v1",
  "data": {
    "items": [
      { "dlq_id": "01JZ8V6H5Q8YJ5HY8D7M2QW1X2", "event_id": "evt_01JZ8V1", "forward_id": "fwd_attempt_01JZ8V2", "forwarder_id": "fwd_01HV9S", "created_at": "2026-06-19T08:33:40Z" }
    ],
    "has_more": false
  }
}