Timeouts, AbortController, and Circuit Breaker (with Retry-After & jittered backoff)

[hoangsonww]

Summary

Extend FastFetch with first-class request timeouts & cancellation, per-endpoint circuit breakers, and standards-aware retry handling (parsing Retry-After, honoring 429/5xx), using exponential backoff with full jitter. This makes retries safer, avoids thundering herds, and gives callers precise control over slow or failing dependencies.

---

Why

• Control slowness: Today retries may wait on hung connections; timeouts + aborts protect UX.
• Be a good citizen: Respect Retry-After and add jitter to reduce coordinated spikes.
• Fail fast: Circuit breakers stop hammering degraded endpoints and provide quick fallback.
• DX: A consistent abstraction over fetch/axios cancellation differences.

---

Proposed API

New options (extend FastFetchOptions)

type FastFetchOptions = {
  // existing
  retries?: number;
  retryDelay?: number;
  deduplicate?: boolean;
  shouldRetry?: (errorOrResponse: Response | Error, attempt: number) => boolean;

  // new
  timeoutMs?: number;                    // per-request timeout (controller-based)
  signal?: AbortSignal;                  // external cancellation
  backoff?: 'fixed' | 'exp' | 'exp-jitter';  // default: 'exp-jitter'
  baseDelayMs?: number;                  // default: 500
  maxDelayMs?: number;                   // cap backoff, default: 30_000
  respectRetryAfter?: boolean;           // default: true
  breaker?: {
    key?: string;                        // e.g., origin or route id (defaults to origin)
    failureThreshold?: number;           // e.g., 0.5 over window
    minSamples?: number;                 // open only after N requests (default 20)
    openWindowMs?: number;               // how long to stay open (default 60_000)
    halfOpenProbeCount?: number;         // probes allowed while half-open (default 1)
  };
};

New exports

// Inspect & control breakers at runtime
getBreakerState(key?: string): { state: 'closed'|'open'|'half-open'; stats: {...} };
resetBreaker(key?: string): void;

---

Behavior Details

• Timeouts & Abort

  • Internally create an AbortController per call; race it with user-provided signal.
  • On timeout, reject with TimeoutError (distinct type) and count as a failure for the breaker.

• Retry policy

  • Default to exponential backoff with full jitter (rand(0, base * 2^attempt) capped by maxDelayMs).
  • If response has Retry-After (HTTP-date or seconds) and respectRetryAfter=true, prefer it over computed backoff (still add small jitter).
  • shouldRetry still consulted last; 4xx other than 408/409/425/429 do not retry by default.

• Circuit Breaker (per key)

  • Closed → Open: Rolling window failure rate exceeds failureThreshold once minSamples reached.
  • Open: Short-circuit requests (reject immediately with CircuitOpenError) until openWindowMs elapses.
  • Half-Open: Allow halfOpenProbeCount trial request(s); success closes breaker, failure re-opens.
  • Failures counted as: network errors, timeouts, 5xx, and 429 with exhausted retries.

• Axios integration

  • Continue deduplication; cancellation uses axios CancelToken/AbortController where available.

---

Examples

import { fastFetch } from '@hoangsonw/fast-fetch';

// 1) Timeout + jittered retries + Retry-After
await fastFetch('/api/report', {
  timeoutMs: 4000,
  retries: 3,
  backoff: 'exp-jitter',
  respectRetryAfter: true
});

// 2) External cancellation (e.g., route change)
const controller = new AbortController();
const p = fastFetch('/api/search?q=lofi', { signal: controller.signal, timeoutMs: 2500 });
controller.abort(); // user navigated away

// 3) Circuit breaker per-origin
await fastFetch('https://slow.example.com/data', {
  breaker: { key: 'slow-origin', failureThreshold: 0.5, minSamples: 30, openWindowMs: 60000 }
});

---

Acceptance Criteria

• timeoutMs reliably aborts the underlying fetch/axios request; callers receive a distinct TimeoutError.
• On 429/503 with Retry-After, FastFetch delays next attempt accordingly (± jitter).
• Backoff defaults to exp-jitter and never exceeds maxDelayMs.
• Circuit breaker transitions (closed/open/half-open) are observable via getBreakerState(); open breaker short-circuits within ≤1ms.
• Unit tests cover: timeout, abort, jitter distribution bounds, Retry-After parsing, breaker transitions, axios + fetch parity.

---

Notes / Future Follow-ups

• Pluggable global rate limiter (token bucket) to cap concurrency across routes.
• Hedged requests for tail-latency reduction (opt-in for idempotent GETs).
• Idempotency-Key helper for POST dedupe where servers support it.