Payment Webhooks Arrive Late, Twice, and Out of Order

A payment provider will tell you that it delivers events reliably. That is true and it is a narrower promise than most teams read it as. What it means is that the event will arrive eventually, probably more than once, possibly hours after the thing it describes, and not necessarily in the order the things happened.

Every design decision below follows from taking that sentence literally.

The redirect is a user interface, not a result

When the customer comes back from the payment page, your application receives a request that says, in effect, the customer is here again. It does not say the payment succeeded, and it is not sent by the payment provider - it is sent by the customer's browser, which is a thing that can be closed, refreshed, or replaced by a phone running out of battery in a lift.

Use the redirect for exactly one job: showing a screen. Read the payment's current state from the provider, render what you know, and if you do not know yet, say so. What the redirect must never do is write "paid" to your database.

The payment is confirmed by the event, and the event arrives on a channel the customer cannot affect.

Verify the bytes, not the object

The signature is a hash over the raw request body with a shared secret, plus a timestamp to stop an old delivery being replayed. Two things go wrong and both look like a wrong key:

public function handle(Request $request): Response
{
    $payload = $request->getContent();   // raw, not $request->all()
 
    try {
        $event = Webhook::constructEvent(
            $payload,
            $request->header('Stripe-Signature'),
            config('services.stripe.webhook_secret'),
        );
    } catch (SignatureVerificationException) {
        return response()->noContent(400);
    }
 
    ProcessPaymentEvent::dispatch($event->id, $payload);
 
    return response()->noContent(200);
}

The first is parsing before hashing. $request->all() gives you an array, and re-encoding that array produces different bytes than arrived - different key order, different unicode escaping, different float formatting. Hash the string.

The second is the CSRF middleware, which will reject the request before any of this runs, because a payment provider has no session and no token. The route belongs outside the web middleware group, and if it is inside, the failure is a 419 that the provider retries for three days.

Note what the handler does after verifying: almost nothing. It hands the work to a queue and returns. Providers expect a fast acknowledgement, and doing the work inline means a slow database query turns into a retry storm.

Out of order is the normal case

Two events about one payment can arrive in the wrong sequence. A charge succeeds and is refunded ninety seconds later; the refund event overtakes the success event; your handler processes the refund against an order that is not yet paid, decides that makes no sense, and does nothing. Now the order is paid forever.

Do not fix this with ordering. Fix it by making each handler describe the world rather than a transition:

• Wrong: "on refund, set status from captured to refunded."
• Right: "on any event for this payment, fetch the payment's current state from the provider and make my row match it."

The event becomes a signal to go and look, and its arrival order stops mattering. It costs one API call per event and removes an entire class of bug that is otherwise found in production by a confused accountant.

Where you genuinely cannot re-fetch, store the provider's event timestamp on the row and ignore anything older than what you have already applied.

Twice is also the normal case

Every provider retries on anything that is not a 2xx, and a delivery that timed out after your code succeeded gets retried too. The same event will land twice.

Store the provider's event id with a unique index, insert it before processing, and let the database reject the duplicate. That is the whole mechanism and it is more reliable than checking whether the row already exists, because a check followed by an insert is a race between two workers, and under retry storms both workers are real.

Then stop trusting the channel entirely

Everything above makes the webhook path correct. It does not make it complete, because an endpoint that was down for four hours during a deploy is an endpoint that missed events, and after the retry window closes they are gone.

So run a reconciliation job. Once a day, ask the provider for every payment that changed since your last successful run and compare it against your rows. Report the differences rather than silently correcting them - a mismatch usually means a bug, and a job that quietly fixes your data will hide that bug for a year.

This is the same shape as the accounting integrations we build: an event stream for freshness, a periodic pull for correctness, and the pull is what you can actually rely on. For payments there is a second reason to want it, which is that somebody in finance is going to ask why the provider's monthly total and your database disagree, and "they do not" is a much better answer than an investigation.

If you are wiring this up alongside the order model itself, how the payment is modelled decides half of what these handlers have to do, and it is worth settling first.