Webhooks

Webhooks enable real-time integration by triggering workflows instantly when events occur in your source systems. Instead of polling APIs on a schedule, webhooks push data to Retrieve the moment something happens.

How Webhooks Work

Webhooks are HTTP callbacks that your source systems send to Retrieve when specific events occur.

Setting Up Webhooks

Step 1: Enable Webhooks for Your Application

In your Retrieve application settings, enable the webhook feature. This automatically provisions a dedicated webhook subdomain for your application.

Your webhook subdomain: https://webhook-yourapp.retrievetl.com
(Automatically provisioned when webhooks are enabled)

Step 2: Configure IP/Domain Whitelist

For security, you must whitelist the IP addresses or domains that are allowed to send webhooks to your application.

Configuration example:

# Environment configuration
RETRIEVE_ALLOWED_LIST=shopify.com,203.0.113.0,198.51.100.0

# Allows webhooks from:
# - Any Shopify domain (*.shopify.com)
# - Specific IP addresses

Step 3: Get Your Integration Webhook URL

Each integration in your application gets a unique webhook URL:

Format: https://webhook-yourapp.retrievetl.com/integration/{INTEGRATION_ID}

Example: https://webhook-yourapp.retrievetl.com/integration/507f1f77bcf86cd799439011

Where to find your Integration ID:

• In the Retrieve dashboard, go to your integration settings
• The integration ID is shown in the URL or in the integration details
• Copy this ID to construct your webhook URL

Step 4: Configure the Source System

In your source system (Shopify, Magento, WooCommerce, etc.), configure the webhook to send events to your Retrieve URL.

Step 5: Test Your Webhook

Most source systems provide a way to test webhooks. Send a test event and monitor in Retrieve to ensure it's received and processed correctly.

Webhook Request Formats

Standard JSON Webhook (Most Common)

Most systems send webhooks as HTTP POST with JSON payload:

// HTTP POST to: https://webhook-yourapp.retrievetl.com/integration/ABC123
// Headers: Content-Type: application/json

{
  "event": "order.created",
  "order": {
    "id": 123456,
    "order_number": "#1001",
    "email": "customer@example.com",
    "total_price": "99.99",
    "currency": "USD",
    "items": [
      {
        "product_id": 789,
        "sku": "TSHIRT-001",
        "quantity": 2,
        "price": "29.99"
      }
    ],
    "shipping_address": {
      "first_name": "John",
      "last_name": "Doe",
      "address1": "123 Main St",
      "city": "New York",
      "zip": "10001"
    }
  },
  "created_at": "2025-12-10T14:30:00Z"
}

XML Webhook Support

Some legacy systems send webhooks in XML format. Retrieve supports this with a special endpoint:

For XML webhooks use: https://webhook-yourapp.retrievetl.com/integration/{ID}/xml

XML payload example:

<?xml version="1.0" encoding="UTF-8"?>
<webhook>
  <event>order.created</event>
  <order>
    <id>123456</id>
    <order_number>#1001</order_number>
    <email>customer@example.com</email>
    <total_price>99.99</total_price>
    <items>
      <item>
        <sku>TSHIRT-001</sku>
        <quantity>2</quantity>
      </item>
    </items>
  </order>
</webhook>

Webhook vs. Scheduled Integration

Real-World Use Cases

Use Case 1: Real-Time Order Processing

Scenario: E-commerce store needs to send new orders to warehouse system immediately

Benefits:

• Warehouse receives orders within seconds of customer checkout
• Faster fulfillment and shipping times
• No delay between order placement and processing

Use Case 2: Customer Signup to CRM

Scenario: New customer signs up on website, immediately added to CRM for marketing

Benefits:

• Instant welcome email sent from CRM
• Customer enters marketing automation immediately
• No gap in customer communication

Use Case 3: Inventory Alerts

Scenario: When product stock drops below threshold, trigger restock workflow

Benefits:

• Immediate notification when stock is low
• Prevent out-of-stock situations
• Automated purchase order creation

How Retrieve Processes Webhooks

When a webhook hits your Retrieve endpoint, here's what happens:

1. Request Received - Webhook server receives the HTTP POST request
2. Security Validation - Verifies the sender is authorized via IP/domain whitelist
3. Integration Lookup - Finds the integration associated with the URL path
4. Job Scheduling - Creates a job and adds it to the queue for processing
5. Response Sent - Immediately returns success or error response to the sender
6. Asynchronous Processing - Job is processed by your integration workflow

This approach ensures fast response times (typically under 100ms) while your integration workflow processes the data asynchronously in the background.

Webhook Response Handling

Success Response

When the webhook is accepted and queued:

{
  "result": true,
  "error": ""
}

Error Response

When the webhook is rejected (unauthorized IP, invalid integration, etc.):

{
  "result": false,
  "error": "This domain/IP (203.0.113.5) isn't authorized on Retrieve"
}

Monitoring & Debugging

Webhook Logs

All webhook requests are logged in your Retrieve dashboard:

• Timestamp - When the webhook was received
• Source IP - Origin of the request
• Integration ID - Which integration was triggered
• Status - Success or failure
• Payload - The data sent (for debugging)
• Processing Result - How the workflow handled the data

Common Issues and Solutions

Best Practices

1. Secure Your Webhooks

• ✅ Always use IP/domain whitelisting
• ✅ Use HTTPS endpoints (Retrieve provides this automatically)
• ✅ Validate webhook signatures if your source system provides them
• ✅ Monitor webhook logs for suspicious activity

2. Handle Webhook Failures Gracefully

• ✅ Use rewrite functions to validate incoming data
• ✅ Set up error notifications for failed webhooks
• ✅ Keep a fallback scheduled job for critical integrations
• ✅ Test webhooks thoroughly before going live

3. Optimize for Performance

• ✅ Keep webhook processing fast - move heavy operations to async
• ✅ Use field mapping instead of complex rewrite functions when possible
• ✅ Batch multiple webhooks if your source sends them in bursts
• ✅ Monitor processing times and optimize slow integrations

4. Plan for Scale

• ✅ Test with high volumes if you expect many webhooks
• ✅ Consider rate limiting on source system if needed
• ✅ Monitor queue depth during peak times
• ✅ Have alerts for webhook processing delays

Advanced: Custom Webhook Processing

Using Rewrite Functions with Webhooks

You can use rewrite functions to customize how webhook data is processed:

// Rewrite function for webhook-triggered integration
let job = jobData.job;
let webhookPayload = job.data.data;

// Webhook might send single object or array
// Ensure we always work with array
let orders = Array.isArray(webhookPayload) ? webhookPayload : [webhookPayload];

// Process webhook data
let processedOrders = orders.map(order => {
  // Validate webhook data
  if (!order.id || !order.email) {
    console.log(`Invalid order data: ${JSON.stringify(order)}`);
    return null;
  }
  
  return {
    order_id: order.id,
    order_number: order.order_number,
    customer_email: order.email,
    total: parseFloat(order.total_price),
    currency: order.currency || 'USD',
    // Add webhook metadata
    webhook_received_at: new Date().toISOString(),
    source: 'webhook'
  };
}).filter(order => order !== null); // Remove invalid orders

return {
  jobStatus: 1,
  data: processedOrders,
  message: `Processed ${processedOrders.length} orders from webhook`
};

Combining Webhooks with Scheduled Jobs

For maximum reliability, combine both approaches:

Example implementation:

• Webhook integration: Processes orders in real-time as they're created
• Scheduled integration: Runs every 6 hours to check for any orders the webhook might have missed
• Result: Fast real-time sync with a safety net for reliability

Next Steps

• Integrations - Learn how webhooks fit into integration workflows
• Rewrite Functions - Customize webhook data processing
• Field Mapping - Transform webhook payloads
• Understanding Workflows - Build multi-step webhook-triggered workflows