Course/Chapter 10/1. Architecture for Scale
    advanced
    Chapter 10: Scaling & Production Best Practices

    1. Architecture for Scale

    Design patterns for high-throughput workflows.

    20m Lesson 1 of 5

    Scaling n8n requires thoughtful architecture. Learn patterns that handle thousands of executions per day without bottlenecks.

    Key Patterns

    • - Event-driven: Decouple producers and consumers with webhooks
    • - Queue-based: Use message queues for reliable, ordered processing
    • - Saga pattern: Coordinate long-running, multi-step processes
    • - Circuit breaker: Prevent cascading failures when services are down

    Workflow Decomposition

    Break monolithic workflows into smaller, focused units:

    1. Ingest workflow: Receives and validates incoming data
    2. Processing workflow: Transforms and enriches data
    3. Output workflow: Delivers results to target systems
    4. Error workflow: Handles failures from all other workflows

    Scaling Checklist

    • - Are you using PostgreSQL (not SQLite) for the database?
    • - Is execution data pruning enabled?
    • - Are large workflows split into sub-workflows?
    • - Do you have queue mode enabled for high concurrency?
    • - Are you monitoring execution queue depth?

    Code Examples

    Circuit Breaker Pattern
    javascript
    // Simple circuit breaker in a Code node
const CIRCUIT_KEY = "api_circuit_breaker";
const THRESHOLD = 5;     // failures before opening
const RESET_MS = 60000;  // 1 minute cooldown

// Read circuit state (use n8n static data)
const state = $getWorkflowStaticData("global");
const failures = state[CIRCUIT_KEY + "_failures"] || 0;
const lastFailure = state[CIRCUIT_KEY + "_lastFail"] || 0;
const isOpen = failures >= THRESHOLD && 
  (Date.now() - lastFailure) < RESET_MS;

if (isOpen) {
  // Circuit is OPEN — skip the API call
  return [{ json: { 
    status: "circuit_open", 
    message: "API temporarily unavailable, retrying later",
    retry_after: new Date(lastFailure + RESET_MS).toISOString()
  }}];
}

// Circuit is CLOSED — proceed with API call
try {
  const result = await $http.request({ method: "GET", url: $json.api_url });
  state[CIRCUIT_KEY + "_failures"] = 0; // Reset on success
  return [{ json: result }];
} catch (error) {
  state[CIRCUIT_KEY + "_failures"] = failures + 1;
  state[CIRCUIT_KEY + "_lastFail"] = Date.now();
  throw error;
}

    Pro Tips

    • 💡Start simple and add complexity only when you hit actual bottlenecks — premature optimization is the enemy
    • 💡Use n8n static data ($getWorkflowStaticData) to maintain state between executions
    • 💡Monitor execution queue depth — if it keeps growing, you need more workers

    Comprehension Quiz

    Answer 2 of 2 correctly to unlock lesson completion.

    1. What pattern prevents cascading failures?

    2. When should you add architectural complexity?

    Pass the quiz above to unlock lesson completion
    PreviousNext Lesson

    All lessons in this chapter

    1. Architecture for Scale20m2. Version Control & CI/CD15m3. Queue Mode & Worker Scaling15m4. Team Collaboration & Access Control15m5. Disaster Recovery & High Availability15m