Building AI Agent on Cloudflare

Build AI agents on Cloudflare using the Agents SDK with state management, real-time WebSockets, scheduled tasks, tool integration, and chat capabilities.

Download this Skill

npx degit LangbaseInc/agent-skills/building-ai-agent-on-cloudflare my-ai-agent-cloudflare

Overview

Build production-ready AI agents that run on Cloudflare's global edge network with Workers AI, Durable Objects, and the Agents SDK.

Architecture

User → Workers (HTTP/WebSocket)
         ↓
    AI Agent (Durable Object)
         ↓
   [Workers AI, KV, D1, R2, Tools]

Key Features

Workers AI

Run LLMs on the edge
100+ models available
No cold starts
Global deployment
Pay per use

Durable Objects

Stateful agents
WebSocket support
Strong consistency
Automatic persistence
Global coordination

Agents SDK

Chat management
Tool integration
Memory/context
Streaming responses
Multi-turn conversations

Quick Start

1. Install Dependencies

npm create cloudflare@latest my-agent
cd my-agent
npm install @cloudflare/ai

2. Configure Worker

export interface Env {
  AI: any;
  AGENT: DurableObjectNamespace;
  KV: KVNamespace;
  DB: D1Database;
}

3. Create Agent

import { WorkersAI } from '@cloudflare/ai';

export class AIAgent {
  constructor(state: DurableObjectState, env: Env) {
    this.state = state;
    this.env = env;
    this.ai = new WorkersAI(env.AI);
  }

  async chat(message: string) {
    const response = await this.ai.run('@cf/meta/llama-3-8b-instruct', {
      messages: [
        { role: 'user', content: message }
      ]
    });
    return response;
  }
}

State Management

Durable Objects Storage

// Save state
await this.state.storage.put('conversation', messages);

// Load state
const messages = await this.state.storage.get('conversation');

// Update state
await this.state.storage.transaction(async (txn) => {
  const current = await txn.get('count');
  await txn.put('count', current + 1);
});

Real-Time WebSockets

export class ChatAgent {
  async fetch(request: Request) {
    const upgradeHeader = request.headers.get('Upgrade');
    if (upgradeHeader === 'websocket') {
      const pair = new WebSocketPair();
      this.handleWebSocket(pair[1]);
      return new Response(null, {
        status: 101,
        webSocket: pair[0]
      });
    }
  }

  async handleWebSocket(ws: WebSocket) {
    ws.accept();

    ws.addEventListener('message', async (event) => {
      const message = event.data;
      const response = await this.chat(message);
      ws.send(JSON.stringify(response));
    });
  }
}

Tool Integration

const tools = {
  getWeather: async (city: string) => {
    const data = await fetch(`https://api.weather.com/${city}`);
    return data.json();
  },

  searchDatabase: async (query: string) => {
    const results = await env.DB.prepare(
      'SELECT * FROM docs WHERE content MATCH ?'
    ).bind(query).all();
    return results;
  }
};

// Use tools in agent
const result = await this.ai.run('@cf/meta/llama-3-8b-instruct', {
  messages: [...],
  tools: Object.keys(tools),
});

if (result.tool_calls) {
  for (const call of result.tool_calls) {
    const output = await tools[call.name](...call.arguments);
    // Continue conversation with tool output
  }
}

Scheduled Tasks

export default {
  async scheduled(event: ScheduledEvent, env: Env) {
    // Run agent tasks periodically
    const agent = env.AGENT.get(id);
    await agent.performScheduledTask();
  }
};

Memory & Context

Short-term Memory

class Agent {
  messages: Message[] = [];

  async chat(message: string) {
    this.messages.push({ role: 'user', content: message });
    const response = await this.ai.run(model, {
      messages: this.messages
    });
    this.messages.push({ role: 'assistant', content: response });
    return response;
  }
}

Long-term Memory (Vectorize)

// Store embeddings
const embedding = await env.AI.run('@cf/baai/bge-base-en-v1.5', {
  text: message
});
await env.VECTORIZE.insert([{
  id: msgId,
  values: embedding,
  metadata: { text: message }
}]);

// Retrieve similar
const results = await env.VECTORIZE.query(embedding, { topK: 5 });

Streaming Responses

async stream(message: string) {
  const stream = await this.ai.run(model, {
    messages: [{ role: 'user', content: message }],
    stream: true
  });

  return new Response(stream, {
    headers: { 'content-type': 'text/event-stream' }
  });
}

Best Practices

Use Durable Objects for state
Implement proper error handling
Cache frequently used data
Monitor token usage
Implement rate limiting
Use WebSockets for real-time
Version your agents
Test thoroughly

Deployment

# Deploy worker
wrangler deploy

# Monitor
wrangler tail

# Check analytics
wrangler pages deployment tail

Use Cases

Customer support chatbots
Code assistants
Content generation
Data analysis agents
Task automation
Research assistants
Interactive tutorials

Skills