> ## Documentation Index
> Fetch the complete documentation index at: https://wemstudios.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Workers AI in Blaze: Text Generation and Streaming

> Call Cloudflare Workers AI models from Blaze route handlers via req.env.AI. Supports text generation, streaming SSE responses, and multiple model families.

Cloudflare Workers AI is a serverless inference API that runs machine learning models on Cloudflare's global GPU network — no model hosting, no infrastructure to manage. Blaze exposes it on `req.env.AI` in every handler once you add the binding to your `Env` type. You get access to a broad catalog of model families — text generation, embeddings, image classification, speech recognition, and more — with optional streaming for real-time output.

## Configure the binding

<Steps>
  <Step title="Add the AI binding to wrangler.toml">
    ```toml wrangler.toml theme={null}
    [ai]
    binding = "AI"
    ```
  </Step>

  <Step title="Define the Env type">
    ```typescript src/types/env.ts theme={null}
    export type Env = {
      AI: Ai;
      // ...other bindings
    };
    ```
  </Step>

  <Step title="Pass Env to createApp">
    ```typescript src/index.ts theme={null}
    import { createApp } from "blaze";
    import type { Env } from "./types/env";

    const app = createApp<Env>();
    // req.env.AI is fully typed as Ai in every handler
    ```
  </Step>
</Steps>

## Text generation

Call `req.env.AI.run(model, inputs)` to run inference synchronously. Pass a `messages` array in the OpenAI chat format and set `stream: false` to receive the complete response in one go.

```typescript theme={null}
app.post("/ai/complete", async (req, res) => {
  const { prompt, systemPrompt } = await req.json<{
    prompt:       string;
    systemPrompt?: string;
  }>();

  const result = await req.env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
    messages: [
      {
        role:    "system",
        content: systemPrompt ?? "You are a helpful assistant.",
      },
      {
        role:    "user",
        content: prompt,
      },
    ],
    stream: false,
  });

  res.json({ response: result.response });
});
```

## Streaming responses

Set `stream: true` to receive a `ReadableStream` of server-sent events. Pipe it directly to the client using `res.stream()` so tokens appear in the browser as they are generated — with no buffering in your Worker.

```typescript theme={null}
app.post("/ai/stream", async (req, res) => {
  const { prompt } = await req.json<{ prompt: string }>();

  const aiStream = await req.env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
    messages: [{ role: "user", content: prompt }],
    stream:   true,
  });

  res.header("Content-Type",  "text/event-stream");
  res.header("Cache-Control", "no-cache");
  res.header("Connection",    "keep-alive");

  res.stream((writer) => aiStream.pipeTo(writer));
});
```

Your client can consume the stream with the standard `EventSource` API or `fetch` with a `ReadableStream` reader:

```typescript theme={null}
const response = await fetch("/ai/stream", {
  method:  "POST",
  headers: { "Content-Type": "application/json" },
  body:    JSON.stringify({ prompt: "Explain Cloudflare Workers in one paragraph." }),
});

const reader = response.body!.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  process.stdout.write(decoder.decode(value));
}
```

## Available models

Workers AI supports multiple model families. Browse the full catalog — including model IDs, input/output schemas, and benchmark scores — at [developers.cloudflare.com/workers-ai/models/](https://developers.cloudflare.com/workers-ai/models/).

| Family                   | Example model                                  | Use case                         |
| ------------------------ | ---------------------------------------------- | -------------------------------- |
| **Text generation**      | `@cf/meta/llama-3.1-8b-instruct`               | Chat, completion, summarization  |
| **Text embeddings**      | `@cf/baai/bge-base-en-v1.5`                    | Semantic search, RAG pipelines   |
| **Image classification** | `@cf/microsoft/resnet-50`                      | Label images by category         |
| **Speech recognition**   | `@cf/openai/whisper`                           | Transcribe audio to text         |
| **Text-to-image**        | `@cf/stabilityai/stable-diffusion-xl-base-1.0` | Generate images from prompts     |
| **Translation**          | `@cf/meta/m2m100-1.2b`                         | Translate between 100+ languages |

```typescript theme={null}
// Embeddings example — generate a vector for semantic search
app.post("/ai/embed", async (req, res) => {
  const { text } = await req.json<{ text: string }>();

  const result = await req.env.AI.run("@cf/baai/bge-base-en-v1.5", {
    text,
  });

  res.json({ embedding: result.data[0] });
});

// Speech recognition example — transcribe an audio file
app.post("/ai/transcribe", async (req, res) => {
  const audioBuffer = await req.arrayBuffer();

  const result = await req.env.AI.run("@cf/openai/whisper", {
    audio: [...new Uint8Array(audioBuffer)],
  });

  res.json({ text: result.text });
});
```

## Error handling

AI inference can fail due to upstream model errors, invalid inputs, or capacity limits. Wrap every `req.env.AI.run()` call in a `try/catch` and throw a `BlazeError(502)` so your global error handler can return a consistent error shape.

```typescript theme={null}
import { BlazeError } from "blaze";

app.post("/ai/complete", async (req, res) => {
  const { prompt } = await req.json<{ prompt: string }>();

  let result: { response: string };

  try {
    result = await req.env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
      messages: [{ role: "user", content: prompt }],
      stream:   false,
    });
  } catch (err) {
    throw new BlazeError(502, "AI inference failed. Please try again.", {
      code: "AI_UPSTREAM_ERROR",
    });
  }

  if (!result.response) {
    throw new BlazeError(502, "AI returned an empty response.", {
      code: "AI_EMPTY_RESPONSE",
    });
  }

  res.json({ response: result.response });
});
```

Pair this with a global error handler so `BlazeError` instances are serialized consistently:

```typescript theme={null}
app.onError((err, req, res, _next) => {
  const status  = err instanceof BlazeError ? err.status  : 500;
  const message = err instanceof BlazeError ? err.message : "Internal Server Error";
  const meta    = err instanceof BlazeError ? err.meta    : {};

  res.status(status).json({ error: message, ...meta });
});
```

<Note>
  Workers AI usage is metered by **Neurons** — Cloudflare's unit of inference
  compute. Costs and rate limits vary by model and plan. Monitor your
  consumption and configure spending limits in the **Cloudflare dashboard →
  Workers AI** section. Streaming responses consume the same number of Neurons
  as non-streaming calls for the same input and output length.
</Note>