Reliability & Timeouts

Production-grade agents must handle hanging remote APIs and slow local tool execution. Chai AI SDK provides built-in timeouts for both model and tool execution to ensure your system fails predictably and gracefully.

Configuring Timeouts

Timeouts can be configured via options on the Agent constructor:

import { Agent } from "chai-ai";
import { google } from "@chai-ai/google";

const agent = new Agent({
  name: "unreliable-ops-agent",
  model: google("gemini-3.1-pro-preview"),
  // Model API timeout (default is 60_000ms)
  modelTimeoutMs: 15_000, 
  // Tool execution timeout (default is 30_000ms)
  toolTimeoutMs: 5_000,
});

1. Model Timeouts (Real Cancellation)

When a model call exceeds modelTimeoutMs, the SDK doesn't just ignore the response; it performs a real network-level cancellation using an AbortController:

• Mechanism: The SDK creates an AbortSignal and forwards it to the provider adapter (doGenerate/doStream), which aborts the connection.
• Provider Support: Natively integrated into @chai-ai/google, @chai-ai/openai, and @chai-ai/anthropic (all three pass the abort signal down to their underlying SDKs).
• Result: The call rejects immediately, propagating to run()'s top-level error handler. The run fails with a run_failed trace entry and throws a standard AgentError.

// Sequence: Model Timeout
agent.generate({ prompt: "..." })
  → model.doGenerate(..., { signal })
  → timer exceeds 15,000ms → controller.abort()
  → request is aborted at the HTTP connection layer
  ← throws AgentError: "Model call timed out"

2. Tool Timeouts (Promise Race Gating)

Unlike model calls, a tool is arbitrary user-defined code. In JavaScript, there is no clean way to force-abort synchronous code or a promise that doesn't natively support abort signals.

• Mechanism: The SDK wraps tool execution in a Promise.race wrapper.
• Graceful Rejection: If the tool takes longer than toolTimeoutMs to resolve, the SDK rejects the wait and treats the timeout as a tool execution error instead of crashing the run.
• Model Loop Recovery: The error { error: "Tool '<toolName>' timed out after <ms>." } is sent back to the model as the tool execution result. The model can see the failure, apologize to the user, and retry or try another approach.

// Sequence: Tool Timeout
Model Call → requests tool: slowTask()
  → runTool(slowTask) wrapped with withTimeout(..., 5000)
  → slowTask runs, but takes 10,000ms
  → at 5000ms, withTimeout rejects
  ← runTool returns tool result: { error: "Tool \"slowTask\" timed out after 5000ms." }
  → Model sees the tool result containing the timeout error
  ← Model responds to user: "The slowTask tool timed out. Would you like me to try again?"