Troubleshooting webhooks
If your integration consumes Tulip webhooks, you should monitor for and respond to failed deliveries. When Tulip can’t deliver an event to your endpoint, the event is re-queued and retried with exponential backoff. If every retry fails, the event is dropped and your system falls out of sync with Tulip.
This guide walks through how to find failing deliveries in the Tulip Admin Console, how to read the logs, and how to diagnose the most common causes.
Related Articles:
View deliveries
You can inspect every delivery attempt Tulip has made for your subscriptions from the Admin Console.
- Log into your Tulip Admin Console.
- Go to Integrations → Webhook Logs in the left-hand side panel.
- Use the filters at the top of the page to narrow down by subscription, event type, or status.
💡 You can trigger a delivery on demand by performing the action the subscription listens for — for example, creating a customer via the Tulip application or API will fire
customerCreate.
View logs
Each row on the Webhook Logs page represents one delivery attempt. The following fields are shown:
| Field | Description |
|---|---|
| Event type | The type of event that triggered the delivery (e.g. customerCreate, orderCreate). |
| Subscription | The subscription that produced the delivery. |
| URL | The endpoint the request was POSTed to. |
| Response code | The HTTP status code returned by your endpoint. A 2xx response is treated as success; any other response (or a timeout) is treated as failure. |
| Attempt | Which delivery attempt this row represents. Tulip delivers the original request and then retries up to 3 more times. |
| Timestamp | When the attempt was made, in Z-normalized RFC3339 format. |
A delivery is considered successful when your endpoint returns any 2xx status code within the response window. Anything else — a 4xx, a 5xx, a network error, or a timeout — counts as a failure and triggers the retry schedule described below.
Responses and retries
If a delivery fails, Tulip retries with exponential backoff:
- 5 seconds
- 20 seconds
- 125 seconds
After three failed retries (four attempts in total), the event is dropped. The subscription itself is not removed, so new events will continue to be delivered — but Tulip does not go back and replay the dropped event. You are responsible for reconciling any data gap, typically by re-fetching the affected resource through the Tulip API using the resource.uuid or resource.externalId from the event payload.
Prioritize fixes
When triaging a list of failing deliveries, work down in this order:
- All attempts exhausted for a resource you care about. Data has been dropped. Reconcile by pulling the latest state of the resource from the Tulip API.
- Consistent failures on a single subscription. Suggests a problem with that subscription’s endpoint or auth config (wrong URL, wrong token, expired cert) rather than a platform-wide issue.
- Occasional timeouts or
5xxresponses. Suggests your endpoint is slow or intermittently unhealthy. If these are growing, treat them as a leading indicator before they become dropped events.
View delivery details
Clicking a row on the Webhook Logs page opens the details for that attempt, including:
| Field | Description |
|---|---|
| Endpoint | The URL the request was sent to. |
| Request headers | The full set of headers Tulip sent, including Authorization, X-Tulip-Hmac-256, and X-Webhook-Version. |
| Request payload | The JSON event body Tulip sent. See Example event object for the schema. |
| Response code | The HTTP status code returned by your endpoint. |
| Response body | The first portion of the body returned by your endpoint, if any. Useful for reading error messages your server emitted. |
| Response time | How long your endpoint took to respond. Tulip waits up to 20 seconds before treating the attempt as a timeout. |
| Attempt / retries left | Which attempt this is and how many retries remain. |
Use the request payload and headers together when reproducing a failure locally — replay the exact JSON body and the X-Tulip-Hmac-256 header against your endpoint to confirm your handler behaves correctly.
Troubleshooting failed deliveries
The table below lists the failure modes integrators run into most often, along with what to check first.
| Symptom | Likely cause and what to check |
|---|---|
401 Unauthorized or 403 Forbidden | The Authentication Method set on the subscription doesn’t match what the endpoint expects. Re-open the subscription in Integrations → Webhook Subscriptions and confirm the method (NONE, BEARER, or BASIC) and the token/credentials. For BEARER, confirm the token has not been rotated on the receiving side. |
404 Not Found | The URL on the subscription is wrong or the route has been removed on your side. Update the subscription URL and trigger a test event. |
405 Method Not Allowed | Your endpoint isn’t accepting POST. Tulip always uses POST for webhook deliveries. |
408 Request Timeout / Tulip-reported timeout | Your endpoint took longer than 20 seconds to respond. Acknowledge the request immediately by returning 2xx, queue the payload, and process it asynchronously. |
413 Payload Too Large | Your endpoint has a request-size limit below what Tulip sent. Raise the limit on your proxy / framework (e.g. Nginx client_max_body_size, Express body-parser limit). |
500, 502, 503, 504 | Your endpoint is erroring or unavailable. Use your own monitoring to confirm the service is healthy, then replay recent events. If a dependency (database, queue, downstream API) is down, your handler should fail fast with 5xx so Tulip retries, rather than absorbing the error and returning 200. |
| HMAC signature mismatch | The payload your endpoint verified with HMAC-SHA256 did not match the X-Tulip-Hmac-256 header. Make sure you are hashing the raw request body (no re-serialization, no trimmed whitespace) with the HMAC key from the subscription. The key is available under Webhook Subscriptions → Display HMAC Key. See HMAC Security. |
| Intermittent network errors / TLS failures | Check that your TLS certificate is valid, not expired, and served with its full intermediate chain. Tulip will not deliver to an endpoint it cannot complete a TLS handshake with. |
| Duplicate events | Retries mean your endpoint can receive the same event more than once. Deduplicate on the event uuid — it is stable across retries. |
| Out-of-order events | Tulip does not guarantee strict ordering across events. Always fetch the latest state of the resource from the Tulip API using resource.uuid or resource.externalId rather than relying on the event payload alone. |
| Missing data after a failure window | After three failed retries the event is dropped. Reconcile by re-fetching affected resources from the Tulip API — the event body includes the identifiers you need. |
Aerial-specific notes
Webhook subscriptions trigger on events in Tulip, but the body of the event only contains identifiers — not the full resource. After receiving an event, fetch the up-to-date resource from the Aerial API using the resource.uuid or resource.externalId from the event payload. The correct endpoint for each event type is listed in Supported Events.
A few Aerial-specific pitfalls to watch for:
- Versioning. Tulip includes the
X-Webhook-Versionheader on every delivery. Aerial itself is versioned separately — make sure the?version=query parameter you use when fetching the resource matches a version your integration supports. Pinning to a known version prevents breaking changes from reaching your handler unannounced. See Versioning. - External IDs may not be set yet. If your integration assigns
externalIds asynchronously, the first few events for a brand-new resource may arrive before yourexternalIdis written. Fall back toresource.uuidwhenexternalIdis missing, and see Automating External ID Assignment. - Rate limits. A burst of webhook events (for example, a bulk import) can cause a burst of API reads back into Aerial. If you hit
429 Too Many Requestswhile fetching resources, back off, retry, and consider batching reads where possible.
Best practices
A few habits that will save you from most webhook incidents:
- Respond fast, process later. Acknowledge the request with
200as soon as you’ve validated the signature, then hand the payload to a queue or background worker. Tulip’s 20-second timeout is generous, but inline processing is the single biggest cause of timeouts in production. - Always verify HMAC. No-auth and bearer auth confirm the caller; HMAC confirms the payload. They are not a substitute for each other.
- Be idempotent. Retries are a feature, not a bug. Dedupe on the event
uuidand make sure replaying an event produces the same outcome as the original. - Log the full delivery on your side. Capture the method, URL, headers, body, and response status for every incoming webhook, at least during initial setup. It makes a later incident tractable.
- Reconcile periodically. Even with perfect webhook handling, it is worth running a scheduled job that pulls the latest resources from the Tulip API and compares against your local copy. It catches both dropped events and bugs in your own handler.
Getting help
If you’ve worked through the list above and a subscription is still failing, open a support ticket with:
- The subscription ID.
- The event type.
- The event
uuidof one failing delivery (visible on the delivery details page). - The response code and response body your endpoint returned, or the timeout observed.
- A timestamp (UTC) of the failing attempt.
Those five pieces of information are enough for Tulip support to correlate the delivery on the platform side and get you a useful answer quickly.