Skip to main content

Handling Failures

Learn how to build resilient webhook integrations that gracefully handle failures and ensure reliable event processing.

Common Failure Scenarios​

Network Issues​

  • Connection timeouts
  • DNS resolution failures
  • Intermittent connectivity

Handler Issues​

  • Application downtime
  • Database connection failures
  • Processing logic errors
  • Memory/resource exhaustion

Data Issues​

  • Malformed JSON payloads
  • Unexpected event structures
  • Missing required fields
  • Invalid data types

Retry Strategies​

Exponential Backoff​

const delay = (attempt) => Math.pow(2, attempt) * 1000; // 1s, 2s, 4s, 8s...

async function retryWebhook(webhook, maxAttempts = 3) {
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    try {
      await processWebhook(webhook);
      return; // Success
    } catch (error) {
      if (attempt === maxAttempts - 1) throw error;
      await new Promise(resolve => setTimeout(resolve, delay(attempt)));
    }
  }
}

Circuit Breaker Pattern​

class CircuitBreaker {
  constructor(threshold = 5, timeout = 60000) {
    this.threshold = threshold;
    this.timeout = timeout;
    this.failureCount = 0;
    this.state = 'CLOSED'; // CLOSED, OPEN, HALF_OPEN
    this.nextAttempt = Date.now();
  }
  
  async call(fn) {
    if (this.state === 'OPEN') {
      if (Date.now() < this.nextAttempt) {
        throw new Error('Circuit breaker is OPEN');
      }
      this.state = 'HALF_OPEN';
    }
    
    try {
      const result = await fn();
      this.onSuccess();
      return result;
    } catch (error) {
      this.onFailure();
      throw error;
    }
  }
}

Error Recovery​

Dead Letter Queues​

const failedEvents = [];

app.post('/webhooks', async (req, res) => {
  try {
    await processWebhook(req.body);
    res.status(200).json({ success: true });
  } catch (error) {
    // Add to dead letter queue for manual processing
    failedEvents.push({
      event: req.body,
      error: error.message,
      timestamp: new Date(),
      retryCount: 0
    });
    
    res.status(500).json({ error: 'Processing failed' });
  }
});

Graceful Degradation​

app.post('/webhooks', async (req, res) => {
  try {
    // Critical processing
    await updateOrderStatus(req.body);
    
    try {
      // Non-critical processing
      await sendEmailNotification(req.body);
    } catch (error) {
      // Log but don't fail the webhook
      console.warn('Email notification failed:', error);
    }
    
    res.status(200).json({ success: true });
  } catch (error) {
    res.status(500).json({ error: 'Critical processing failed' });
  }
});

Monitoring and Alerting​

Health Checks​

app.get('/health', (req, res) => {
  const health = {
    status: 'healthy',
    database: checkDatabaseConnection(),
    memory: process.memoryUsage(),
    uptime: process.uptime()
  };
  
  res.json(health);
});

Error Tracking​

const Sentry = require('@sentry/node');

app.post('/webhooks', async (req, res) => {
  try {
    await processWebhook(req.body);
    res.status(200).json({ success: true });
  } catch (error) {
    Sentry.captureException(error, {
      tags: {
        component: 'webhook-handler'
      },
      extra: {
        webhookData: req.body
      }
    });
    
    res.status(500).json({ error: 'Processing failed' });
  }
});

This section is under development. More failure handling patterns coming soon.