Field Mapping Guide

Field mapping is the bridge between different systems. It transforms data from one structure to another, ensuring information flows smoothly between your applications. Each node in your workflow can have its own field mapping configuration.

What is Field Mapping?

Imagine you're translating between two languages - the source system speaks one way, the destination system speaks another. Field mapping is your translator.

Two Approaches to Field Mapping

Retrieve offers two powerful approaches. Think of them as different tools for different jobs:

1. Data Mapping (Field-by-Field): Like using labels - point this field to that field, add a function if needed. Perfect for most cases.
2. After Origin Function (Complete Control): Like having a master craftsman - receive all the data, reshape it however you want. For complex scenarios.

Approach 1: Data Mapping (Recommended for Most Cases)

Data mapping allows you to define mappings at the field level. It's intuitive, visual, and handles 90% of integration scenarios. You configure each destination field individually using the UI.

1.1 Direct Field Mapping

The simplest form - take a field from the source and put it in a field at the destination, no changes:

{
  "customer_email": "email",
  "customer_phone": "phone",
  "product_sku": "sku"
}

How to read this: The destination field customer_email gets its value from the source field email.

1.2 Static Values

Sometimes a destination field needs the same value every time, regardless of source data. This is common for:

• Default statuses: All new customers start as "active" (status = 1)
• Constants: Your warehouse location is always "US-EAST"
• Fixed categories: All imported products go into "E-Commerce" category

{
  "customer_email": "email",
  "status": 1,
  "country_code": "US",
  "currency": "USD"
}

Use case: The destination requires status and country_code fields, but your source doesn't provide them. Set them to static values!

1.3 JavaScript Functions for Field Transformation

When you need to modify a value before sending it, add JavaScript code to that specific field. Perfect for:

• Format changes: "published" → 1, "draft" → 0
• Calculations: Price * quantity = total
• Concatenation: Combine first_name + last_name
• Conditional logic: If X then Y, else Z

Available Variables in Field Functions

Your JavaScript function automatically has access to three powerful variables:

// If mapping "status" field
originalValue = "published"

originalData = {
  status: "published",
  price: 29.99,
  compare_at_price: 39.99,
  title: "T-Shirt"
}

integrationConfig = {
  default_warehouse: "US-EAST",
  default_discount: 10,
  tax_rate: 0.08
}

Example 1: Simple Transformation

Scenario: Source has status as text ("published"/"draft"), destination needs numbers (1/0)

if (originalValue === "published") {
  return 1;
} else {
  return 0;
}

Example 2: Using All Variables Together

Scenario: Calculate discount percentage using original price, compare price, and fallback to config default

// Example: Calculate discount percentage
// originalValue = current price
// originalData = { compare_at_price: 100, price: 80, ... }
// integrationConfig = { default_discount: 10, ... }

if (originalData.compare_at_price && originalValue) {
  const discount = ((originalData.compare_at_price - originalValue) / originalData.compare_at_price) * 100;
  return Math.round(discount);
} else {
  return integrationConfig.default_discount || 0;
}

Example 3: Combining Fields

Scenario: Combine first name and last name into full name

// Combine first and last name
return originalData.first_name + " " + originalData.last_name;

Example 4: Conditional Status Mapping

Scenario: Map different status values to destination's status codes

// Map different status values
if (originalValue === "published") {
  return "active";
} else if (originalValue === "draft") {
  return "inactive";
} else if (originalValue === "archived") {
  return "discontinued";
} else {
  return "unknown";
}

1.4 Complete Data Mapping Example

Real-World Scenario: Syncing Shopify products to an ERP system

Let's map a product with multiple field types:

{
  "sku": "variants[0].sku",
  "product_name": "title",
  "warehouse_location": "US-WEST",
  "status": 1,
  "is_available": "function:return originalValue === 'active' ? true : false;",
  "discount_percentage": "function:if (originalData.compare_at_price && originalValue) { const discount = ((originalData.compare_at_price - originalValue) / originalData.compare_at_price) * 100; return Math.round(discount); } else { return 0; }",
  "full_description": "function:return originalData.title + ' - ' + (originalData.body_html || 'No description available');"
}

What's happening here:

• sku: Direct mapping from source
• product_name: Direct mapping
• warehouse_location: Static value - always "US-WEST"
• status: Static number - always 1 (active)
• is_available: JavaScript function converts "active" text to boolean
• discount_percentage: JavaScript calculates discount from two price fields
• full_description: JavaScript combines title and body_html with formatting

Approach 2: After Origin Function (Advanced Control)

The After Origin Function gives you complete control over data transformation. Instead of mapping field-by-field, you receive ALL the data at once and return whatever structure you need.

2.1 How It Works

2.2 Available Variables

The After Origin Function receives the jobData object with these properties:

// Access the data
job.data.data // Array of records

// Access config
integration.config.default_warehouse

// Use helpers
integrationHelper.formatDate(date)

// Manage queue
queueManager.addToQueue(data)

2.3 Basic Example: Order Transformation

Scenario: Pull orders from Shopify, calculate totals, and format for destination

{
  "increment_id": "12345",
  "customer": {
    "email": "john@example.com",
    "first_name": "John",
    "last_name": "Doe"
  },
  "items": [
    { "price": 29.99, "quantity": 2 },
    { "price": 15.50, "quantity": 1 }
  ]
}

{
  "id": "12345",
  "customer_email": "john@example.com",
  "customer_name": "John Doe",
  "total": 75.48
}

The Function:

const data = job.data.data;

return {
  id: data.increment_id,
  customer_email: data.customer.email,
  customer_name: data.customer.first_name + " " + data.customer.last_name,
  total: data.items.reduce((sum, item) => {
    return sum + (item.price * item.quantity);
  }, 0)
}

2.4 Advanced Example: Complex Multi-Record Transformation

Scenario: Pull orders, filter out cancelled ones, calculate totals, and format customer data

const data = job.data.data;

// Filter out cancelled orders
const activeOrders = data.filter(order => order.status !== 'cancelled');

// Transform each order
const transformedOrders = activeOrders.map(order => {
  // Calculate item total
  const itemTotal = order.items.reduce((sum, item) => {
    return sum + (item.price * item.quantity);
  }, 0);
  
  return {
    order_id: order.increment_id,
    customer: {
      email: order.customer.email,
      name: `${order.customer.first_name} ${order.customer.last_name}`,
      phone: order.customer.phone || 'N/A'
    },
    subtotal: itemTotal,
    tax: order.tax_amount || 0,
    shipping: order.shipping_amount || 0,
    total: itemTotal + (order.tax_amount || 0) + (order.shipping_amount || 0),
    status: order.status === 'complete' ? 'fulfilled' : 'pending',
    item_count: order.items.length
  };
});

return transformedOrders;

What this function does:

1. Filters: Removes cancelled orders from the array
2. Transforms: Maps each order to a new structure
3. Calculates: Computes subtotal from line items
4. Combines: Merges first and last name into full name
5. Restructures: Nests customer data into an object
6. Returns: Clean array ready for the next node

2.5 Real-World Example: Product Enrichment

Scenario: Pull products, add calculated fields, group by vendor, add stock status

const products = job.data.data;

// Group by vendor and enrich with calculations
const enrichedProducts = products.map(product => {
  const variant = product.variants[0] || {};
  const hasDiscount = variant.compare_at_price && variant.compare_at_price > variant.price;
  
  return {
    sku: variant.sku,
    title: product.title,
    vendor: product.vendor,
    price: variant.price,
    compare_price: variant.compare_at_price,
    discount_amount: hasDiscount ? (variant.compare_at_price - variant.price) : 0,
    discount_percent: hasDiscount ? 
      Math.round(((variant.compare_at_price - variant.price) / variant.compare_at_price) * 100) : 0,
    stock_status: variant.inventory_quantity > 0 ? 'in_stock' : 'out_of_stock',
    low_stock_alert: variant.inventory_quantity < 10,
    categories: product.tags || [],
    last_updated: new Date().toISOString()
  };
});

return enrichedProducts;

When to Use Each Approach

Use Data Mapping When:

• ✅ You need simple field-to-field transformations
• ✅ Each destination field maps to one or more source fields
• ✅ Transformations are independent per field
• ✅ You want to leverage static values for certain fields
• ✅ Field-level transformations are sufficient
• ✅ You prefer visual/UI-based configuration
• ✅ Your team has varying technical skill levels

Use After Origin Function When:

• ✅ You need to restructure the entire data array
• ✅ You need to filter records based on complex criteria
• ✅ You need to aggregate or calculate values across multiple records
• ✅ You need to create new records or split existing ones
• ✅ Complex business logic that spans multiple fields or records
• ✅ You need access to integration configuration or helpers
• ✅ You need full programmatic control
• ✅ Data structure transformation is complex

Nested Field Access

Most APIs return nested data structures. Here's how to access them in both approaches:

Understanding Nested Structures

API data often looks like this:

{
  "id": "12345",
  "customer": {
    "email": "john@example.com",
    "address": {
      "city": "New York",
      "country": "US"
    }
  },
  "items": [
    { "sku": "TSHIRT-001", "price": 29.99 },
    { "sku": "JEANS-002", "price": 59.99 }
  ]
}

Accessing Nested Fields in Data Mapping

Use dot notation for objects and array indices for arrays:

{
  "customer_email": "customer.email",
  "city": "customer.address.city",
  "first_item_sku": "items[0].sku",
  "first_item_price": "items[0].price",
  "country": "customer.address.country"
}

Accessing Nested Fields in After Origin Function

Use JavaScript object/array syntax:

const data = job.data.data;

return {
  customer_email: data.customer.email,
  city: data.customer.address.city,
  first_item: data.items[0].sku,
  item_count: data.items.length,
  // Safe access with optional chaining
  backup_email: data.customer?.backup_contact?.email || 'none'
}

Real-World Integration Examples

Example 1: E-Commerce to Accounting

Scenario: Sync Shopify orders to NetSuite for invoicing

Data Mapping Configuration:

{
  "order_number": "increment_id",
  "customer_email": "customer.email",
  "customer_name": "function:return originalData.customer.first_name + ' ' + originalData.customer.last_name;",
  "subtotal": "subtotal_price",
  "tax_amount": "total_tax",
  "shipping_cost": "shipping_lines[0].price",
  "total_amount": "total_price",
  "currency": "currency",
  "order_status": "function:if (originalValue === 'fulfilled') { return 'completed'; } else if (originalValue === 'pending') { return 'processing'; } else { return 'on_hold'; }",
  "payment_method": "payment_gateway_names[0]",
  "shipping_method": "shipping_lines[0].title",
  "warehouse_code": "US-CENTRAL"
}

Example 2: CRM to Email Marketing

Scenario: Send new Salesforce leads to Mailchimp campaigns

{
  "email_address": "Email",
  "first_name": "FirstName",
  "last_name": "LastName",
  "phone": "Phone",
  "company": "Company",
  "lead_source": "LeadSource",
  "status": "function:return originalValue === 'Qualified' ? 'subscribed' : 'pending';",
  "tags": "function:return [originalData.Industry, originalData.LeadSource].filter(Boolean).join(',');",
  "merge_fields": "function:return { COMPANY: originalData.Company, PHONE: originalData.Phone };"
}

Example 3: Multi-Channel Inventory Sync

Scenario: Update inventory across Shopify, Amazon, and eBay from central warehouse

const inventory = job.data.data;

// Transform warehouse data for multiple channels
return inventory.map(item => ({
  sku: item.sku,
  shopify_quantity: item.available_quantity,
  amazon_quantity: item.available_quantity - item.reserved_quantity,
  ebay_quantity: Math.max(item.available_quantity - 5, 0), // Safety buffer
  location: item.warehouse_location,
  last_updated: new Date().toISOString(),
  // Flag for attention
  needs_restock: item.available_quantity < item.reorder_point
}));

Best Practices

1. Choose the Right Approach

• Start with Data Mapping for most scenarios - it's simpler and more maintainable
• Only use After Origin Function when you truly need to restructure/filter entire data sets
• Don't overcomplicate - if Data Mapping works, use it!

2. Keep Functions Focused and Simple

• Each field function should handle one transformation concern
• Avoid complex nested logic - break it into multiple fields if needed
• Use descriptive variable names in After Origin Functions

3. Always Handle Edge Cases

• ✅ Check for null/undefined values before accessing them
• ✅ Provide fallback/default values when data might be missing
• ✅ Validate data types match expectations
• ✅ Handle empty arrays and objects gracefully

// Example of proper edge case handling
if (!originalValue || originalValue.trim() === "") {
  return "N/A"; // Handle empty strings
}

if (typeof originalValue !== "string") {
  return String(originalValue); // Ensure it's a string
}

return originalValue.toUpperCase();

4. Test with Real Data

• Never assume data structure - always test with actual API responses
• Test with edge cases: empty values, special characters, large numbers
• Test with multiple records to ensure consistency
• Use the preview/test feature in the UI before going live

5. Document Complex Logic

• Add comments explaining non-obvious transformations
• Document why certain mappings exist (business requirements)
• Keep a changelog of mapping changes for auditing

// Business requirement: Customers from EU need special handling
// See ticket #1234 for details
if (originalData.country_code && ["DE", "FR", "IT", "ES"].includes(originalData.country_code)) {
  return "EU-WAREHOUSE";
} else {
  return integrationConfig.default_warehouse;
}

6. Use Static Values Wisely

• Perfect for required fields with constant values
• Good for default statuses or categories
• Consider using config instead for values that might change

7. Optimize Performance

• In After Origin Functions, avoid unnecessary loops
• Don't make API calls inside field functions
• Filter early to reduce data processing load
• Use built-in JavaScript methods (map, filter, reduce) efficiently

8. Return Consistent Data Structures

• After Origin Functions must return the expected format
• Ensure field names match what the next node expects
• Keep data types consistent (don't return string sometimes, number other times)

Common Field Mapping Patterns

Pattern 1: Concatenating Multiple Fields

Combine separate fields into one:

// Combine first and last name
return originalData.first_name + " " + originalData.last_name;

Pattern 2: Conditional Value Mapping

Map different input values to corresponding output values:

// Map status values
if (originalValue === "published") {
  return "active";
} else if (originalValue === "draft") {
  return "inactive";
} else {
  return "archived";
}

Pattern 3: Using Configuration Fallbacks

Use a default value from config when source field is empty:

// Use configuration value as fallback
return originalValue || integrationConfig.default_warehouse || "MAIN";

Pattern 4: Array to String Conversion

Convert an array of values into a comma-separated string:

// Convert array to comma-separated string
return originalValue.join(", ");

Pattern 5: Date Formatting

Convert date formats between systems:

// Convert ISO date to MM/DD/YYYY
const date = new Date(originalValue);
const month = String(date.getMonth() + 1).padStart(2, '0');
const day = String(date.getDate()).padStart(2, '0');
const year = date.getFullYear();
return `${month}/${day}/${year}`;

Pattern 6: Price Calculations

Calculate values from multiple numeric fields:

// Calculate total from price, tax, and shipping
const price = parseFloat(originalData.price) || 0;
const tax = parseFloat(originalData.tax) || 0;
const shipping = parseFloat(originalData.shipping) || 0;
return price + tax + shipping;

Pattern 7: Null/Undefined Handling

Safely handle missing or null values:

// Safely handle null values with defaults
return originalValue !== null && originalValue !== undefined 
  ? originalValue 
  : "Default Value";

Troubleshooting Common Issues

// Bad
return originalData.customer.address.city;

// Good
return originalData?.customer?.address?.city || "Unknown";

return parseFloat(originalValue) || 0;

// Bad - no return
const data = job.data.data.map(...);

// Good - return the result
const data = job.data.data.map(...);
return data;

const date = new Date(originalValue);
return date.toISOString().split("T")[0]; // Returns YYYY-MM-DD

Next Steps

• Rewrite Functions - Add filtering and validation logic to your nodes
• Actions (Pull/Push) - Understand how field mapping fits in the data flow
• Understanding Workflows - See how mapping connects nodes in workflows
• Integration Packages - Browse platform-specific field examples