A webhook is an HTTP callback. When something happens in your system, such as a payment succeeding, your server sends a POST request with event data to a URL your customer or partner has registered, so they can react instantly without polling.

How do I secure webhooks?

Sign each payload with a shared secret using HMAC, include a timestamp, and let receivers verify both. Send over HTTPS only, rotate secrets periodically, and reject old timestamps to prevent replay attacks.

How many times should a webhook be retried?

Use exponential backoff over a period of hours to a few days, for example retrying after one minute, then several minutes, then longer gaps. Stop after a defined limit, mark the endpoint unhealthy, and notify the customer.

Can the same webhook be delivered twice?

Yes. At-least-once delivery is the norm, so duplicates will happen. Include a unique event ID in every payload and tell receivers to ignore events they have already processed.

Should webhooks carry full data or just an ID?

Both patterns work. Thin events with an ID are safer and smaller, and the receiver fetches current data from your API. Fat events are convenient but can go stale or leak more data than intended, so choose deliberately.

Webhook Design for SaaS: Retries, Signatures, and Reliable Delivery

Why webhooks decide whether integrations feel trustworthy

Almost every B2B product eventually needs to tell other systems what happened. An invoice was paid, a user signed up, a shipment left the warehouse, a form was submitted. Polling an API every few seconds is wasteful for everyone, so webhooks became the standard: your server calls the customer's URL the moment an event occurs.

Webhooks look simple. They are just an HTTP POST. But the details decide whether customers trust your platform or quietly build workarounds. Messages get lost, arrive twice, arrive out of order, or can be forged by anyone who guesses the URL. This guide covers how to design webhooks that are secure, observable, and forgiving when things go wrong.

If you are earlier in your journey and still planning integrations, our overview of the event-driven architecture for startups shows where webhooks fit alongside internal events.

A real-world example: the missed payment event

Picture a subscription product that notifies customers' accounting tools when an invoice is paid. One afternoon the customer's server is down for a deploy. The webhook fails once and is never retried. The accounting system never learns about the payment, the customer's books drift out of sync, and they open a support ticket asking whether your product is reliable.

None of this is dramatic, but it is exactly how integration trust erodes. A good webhook system would have retried the delivery with backoff, shown the failed attempt in a dashboard, and let the customer replay the event with one click once their server recovered.

Design principles to start from

Assume failure. Receivers will be down, slow, or buggy. Delivery must tolerate that.
Assume duplicates. At-least-once delivery means the same event may arrive twice.
Assume forgery. A public URL can be called by anyone, so every request must be verifiable.
Assume change. Payload formats will evolve, so version them from the start.
Make it visible. Customers should see what was sent, when, and what the response was.

Step-by-step: building a dependable webhook system

Define your event catalogue. List each event with a clear name such as invoice.paid, a short description, and an example payload.
Let customers register endpoints. Provide a settings screen and API to add URLs, choose event types, and see a signing secret.
Write events to a durable store first. Save the event in your database in the same transaction as the business change, then deliver asynchronously.
Deliver from a queue. Workers pick up pending deliveries, send the POST with a short timeout, and record the result.
Sign every request. Compute an HMAC of the timestamp and body with the endpoint's secret and send it in a header.
Retry with backoff. On timeouts or non-2xx responses, schedule later attempts with growing gaps and a sensible cap.
Disable broken endpoints gracefully. After repeated failures, pause the endpoint and email the account owner.
Provide logs and replay. Show delivery history and let customers resend any event.

Step three is the one teams most often skip, and it is the most important. Sending the webhook inline during the request means a crash between the database commit and the HTTP call can lose the event forever. Writing the event first, often called an outbox pattern, closes that gap. Our guide to background jobs and queues explains the worker side in detail.

Security: signatures and replay protection

Sign the payload

Give each endpoint its own secret. For every request, calculate an HMAC (commonly SHA-256) over the timestamp plus the raw request body and put the result in a header. The receiver recomputes it and compares using a constant-time check. If it matches, the request came from you and was not modified.

Include a timestamp

Signing alone does not stop an attacker from capturing a valid request and sending it again later. Including a timestamp in the signed data lets receivers reject anything older than a few minutes.

Use HTTPS and protect your own network

Only deliver to HTTPS URLs. Also guard against server-side request forgery: if customers can register any URL, someone may point it at your internal services. Block private IP ranges, resolve addresses safely, and run delivery workers in a restricted network zone. Teams building the wider security posture may also like our guide to secrets management for startups, which covers storing the signing secrets properly.

Retries, ordering, and idempotency

Retries are what make webhooks reliable, but they also create duplicates and reordering. Plan for both.

Backoff schedule: for example, retry after a minute, then several minutes, then hours, over a day or two. Add random jitter so thousands of retries do not land together.
Event IDs: include a unique ID so receivers can deduplicate.
Ordering: do not promise strict order unless you truly guarantee it. Include a timestamp or sequence number and tell receivers to treat each event as a hint to fetch current state.
Timeouts: give receivers a short window, perhaps a few seconds, and tell them to acknowledge fast and process later.
Response handling: treat any 2xx as success, and retry on timeouts, 5xx, and 429.

On the receiving side, the same discipline applies. Store processed event IDs and skip repeats. A unique constraint on the event ID in a processed events table is usually all that is needed.

Thin events or fat events?

A thin event carries only the event type and an object ID. The receiver calls your API to get the latest data. This keeps payloads small, avoids stale data, and reduces the risk of leaking sensitive fields to a misconfigured endpoint. A fat event includes the full object, which saves a round trip and is handy for simple integrations.

Many teams choose a middle path: a modest payload with the essentials and a link to the full resource. Whatever you pick, document it and keep it consistent.

Versioning and documentation

Once customers depend on a payload, changing it can break their systems silently. The same compatibility thinking applies to mobile clients, as our guide to API versioning and backward compatibility explains. Add a version field to each event, only add new fields in a backward compatible way, and give customers time and a changelog before removing anything. Publish a clear reference with sample payloads, header names, retry rules, and a test tool that sends a sample event on demand.

Observability for you and your customers

Treat webhooks as a product surface. Useful features include a delivery log per endpoint, the exact request and response for each attempt, a manual resend button, and an alert when an endpoint keeps failing. Internally, track success rate, latency, and queue depth. When delivery slows, you want to know before customers do.

Testing webhooks before customers do

Reliable delivery is easy to claim and hard to prove, so build a test routine. Create a small mock receiver that can be told to respond slowly, return errors, or drop the connection. Run your delivery workers against it and watch the retry schedule play out. Confirm that the event appears in the delivery log with the right status, that the signature verifies with a sample receiver in at least two languages, and that a replayed event carries the same ID as the original.

Also test the unhappy configuration paths. What happens when a customer registers a URL that redirects, uses a self-signed certificate, or resolves to a private address? Decide your policy for each, document it, and return a clear message when the endpoint is saved rather than failing silently later. A one-click Send test event button in your dashboard gives customers the same confidence.

Finally, rehearse an incident. If your queue backs up for an hour, can you drain it without flooding receivers? Rate limit deliveries per endpoint so a recovery burst does not knock over a customer's server, which would simply restart the cycle of failures.

Key benefits of doing it well

Customers integrate faster because behaviour is predictable.
Fewer support tickets about missing or duplicated events.
Stronger security posture against forged or replayed requests.
Easier scaling, since delivery is decoupled from user requests.
A genuine selling point for B2B buyers who evaluate integration quality.

Mistakes we see often

Sending webhooks inside the request thread with no queue or retry.
No signature, or signing only part of the payload.
Retrying forever, or never.
Forgetting to give customers visibility into failures.
Changing payload shapes without versioning.
Not testing against a slow or failing receiver.

Build or buy?

Several managed services handle webhook delivery, signing, and retries, which can be a good choice for a small team. Building in-house gives more control and fewer vendors, but the surface is larger than it looks. Weigh it like any platform decision. If you want a second opinion, our SaaS development team helps product companies design integration layers that scale.

Conclusion

Webhooks are a promise: when something happens, we will tell you, and you can trust what we tell you. Keep that promise by writing events durably before sending, signing every request, retrying with backoff, expecting duplicates, versioning payloads, and giving customers visibility and replay.

The first version can be modest. A queue, a signature, a retry schedule, and a delivery log already put you ahead of many products. Build those foundations early and your integrations will become an asset instead of a support burden.

Frequently Asked Questions

What is a webhook?: A webhook is an HTTP callback. When something happens in your system, such as a payment succeeding, your server sends a POST request with event data to a URL your customer or partner has registered, so they can react instantly without polling.
How do I secure webhooks?: Sign each payload with a shared secret using HMAC, include a timestamp, and let receivers verify both. Send over HTTPS only, rotate secrets periodically, and reject old timestamps to prevent replay attacks.
How many times should a webhook be retried?: Use exponential backoff over a period of hours to a few days, for example retrying after one minute, then several minutes, then longer gaps. Stop after a defined limit, mark the endpoint unhealthy, and notify the customer.
Can the same webhook be delivered twice?: Yes. At-least-once delivery is the norm, so duplicates will happen. Include a unique event ID in every payload and tell receivers to ignore events they have already processed.
Should webhooks carry full data or just an ID?: Both patterns work. Thin events with an ID are safer and smaller, and the receiver fetches current data from your API. Fat events are convenient but can go stale or leak more data than intended, so choose deliberately.