Getting Started with the Vercel AI SDK in Node.js

If you've been building AI features in Node.js for a while, you've probably done the "raw fetch to OpenAI" thing after you've gotten tired of the OpenAI npm package.

1. Manually construct messages
2. POST to /v1/chat/completions
3. Parse choices[0].message.content

This works well until you want to support multiple providers - what if you also want to support Claude or Gemini? Not only do Claude and Gemini have different API structures and permissions, but they also have different formats for the messages array. Writing an integration that worked with all of them is a daunting task.

That's exactly where we found ourselves with Mongoose Studio. We rewrote the code to use the Vercel AI SDK instead (PR here) and the result was a much simpler, cleaner abstraction that supports OpenAI, Claude, and Gemini with minimal code.

Introducting the Vercel AI SDK

Vercel's AI SDK gives you a unified interface for different LLM providers. They have a core ai package, and then separate integration packages for different LLM providers, like @ai-sdk/openai and @ai-sdk/anthropic.

To install:

npm install ai @ai-sdk/openai @ai-sdk/anthropic

A Minimal callLLMWrapper()

Here's the core abstraction that we came up with for our refactor (non-streaming). The @ai-sdk/openai package exports a createOpenAI() function you can use to create an OpenAI provider, and there's an analogous createAnthropic() for Claude. The core ai package has a generateText() function that takes in a model (which includes the provider), a system prompt, and an array of messages.

import { createAnthropic } from '@ai-sdk/anthropic';
import { createOpenAI } from '@ai-sdk/openai';
import { generateText } from 'ai';

module.exports = async function callLLM(messages, system, options) {
  let provider = null;
  let model = null;

  if (options?.openAIAPIKey && options?.anthropicAPIKey) {
    throw new Error('Cannot set both OpenAI and Anthropic API keys');
  }

  if (options?.openAIAPIKey) {
    provider = createOpenAI({ apiKey: options.openAIAPIKey });
    model = options?.model ?? 'gpt-4o-mini';
  } else if (options?.anthropicAPIKey) {
    provider = createAnthropic({ apiKey: options.anthropicAPIKey });
    model = options?.model ?? 'claude-haiku-4-5-20251001';
  } else {
    throw new Error('Must set either OpenAI key or Anthropic API key');
  }

  return generateText({
    model: provider(model),
    system,
    messages
  });
};

You can then call callLLM() as follows:

const res = await callLLM(
  [{    
    role: 'user',
    content: 'reverse a string' 
  }],
  'You are an assistant that writes JavaScript programs like a pirate',
  { openAIAPIKey: process.env.OPENAI_API_KEY }
);
// Outputs "Arrr, matey! Here’s a simple JavaScript function to reverse a string:..."
console.log(res.text);

Streaming Responses

generateText() is fine for small text outputs, like text summarization, but for more sophisticated cases where the LLM is expected to generate a lot of text, streaming is the way to go. Otherwise, your user won't see any text until the entire response is done. For example, in the ChatGPT UI, you don't wait for the full response, you see the text appear as the model generates it.

The Vercel AI SDK exposes that via streamText(), which returns chunks of text as they’re produced instead of waiting for the full response. Specifically, streamText() returns an async iterator. If you're surfacing the result of the text generation to the user, streaming makes perceived latency drop dramatically.

For Mongoose Studio, I implemented a separate streamLLM() helper as follows. streamLLM() is an async generator function.

export default async function* streamLLM(messages, system, options) {
  let provider = null;
  let model = null;
  if (options?.openAIAPIKey && options?.anthropicAPIKey) {
    throw new Error('Cannot set both OpenAI and Anthropic API keys');
  }

  if (options?.openAIAPIKey) {
    provider = createOpenAI({ apiKey: options.openAIAPIKey });
    model = options?.model ?? 'gpt-4o-mini';
  } else if (options?.anthropicAPIKey) {
    provider = createAnthropic({ apiKey: options.anthropicAPIKey });
    model = options?.model ?? 'claude-haiku-4-5-20251001';
  } else {
    throw new Error('Must specify either OpenAI or Anthropic key');
  }

  const { textStream } = streamText({
    model: provider(model),
    system,
    messages
  });
  for await (const chunk of textStream) {
    yield chunk;
  }
  return;
}

Here's what using streamLLM() looks like. The easiest way to iterate through an async iterator is with a for/await/of loop.

const stream = streamLLM(
  [{
    role: 'user',
    content: 'reverse a string'
  }],
  'You are an assistant that writes JavaScript programs like a pirate',
  { openAIAPIKey: process.env.OPENAI_API_KEY }
);
// Prints out "Arrr! Here be a JavaScript function to reverse a string, matey!..." chunk by chunk
for await (const chunk of stream) {
  process.stdout.write(chunk);
}

Structured Outputs with generateObject()

So far we've looked at:

• generateText(): returns a string
• streamText(): streams a string

But in real applications, you often don't actually want text. You want structured data. And relying on generateText() to produce valid JSON or YAML that you can parse is brittle, even if you tell the LLM to "please return JSON only or I'll lose my job."

generateObject() fixes this. Put in a prompt and a schema, and you get back an object that matches the schema. The schema must be defined using Zod as of this writing.

import { createAnthropic } from '@ai-sdk/anthropic';
import { createOpenAI } from '@ai-sdk/openai';
import { generateObject } from 'ai';
import z from 'zod';

export default async function myGenerateObject(prompt, schema, options) {
  let provider = null;
  let model = null;
  if (options?.openAIAPIKey && options?.anthropicAPIKey) {
    throw new Error('Cannot set both OpenAI and Anthropic API keys');
  }

  if (options?.openAIAPIKey) {
    provider = createOpenAI({ apiKey: options.openAIAPIKey });
    model = options?.model ?? 'gpt-4o-mini';
  } else if (options?.anthropicAPIKey) {
    provider = createAnthropic({ apiKey: options.anthropicAPIKey });
    model = options?.model ?? 'claude-haiku-4-5-20251001';
  } else {
    throw new Error('Must specify either OpenAI or Anthropic key');
  }

  const result = await generateObject({
    model: provider(model),
    schema,
    prompt
  });
  return result.object;
}

You can then call myGenerateObject() as follows. Yes, modern LLMs are pretty good at getting coordinates of major cities.

// Prints `{ lat: 43.615, lng: -116.2023 }` approximately
console.log(
  await myGenerateObject(
    'Get me the coordinates of Boise, Idaho',
    z.object({ lat: z.number(), lng: z.number() }),
    { openAIAPIKey: process.env.OPENAI_API_KEY }
  )
);

Moving On

The Vercel AI SDK gives you numerous neat primitives for working with different LLM providers while abstracting away the underlying provider. The above examples work fine with OpenAI, Anthropic, and Gemini models in my experience, and that's far from all the providers that the AI SDK supports. The AI SDK also has support for agents, memory, and all sorts of other amazing features. Try it out next time you find yourself wanting to add yet another LLM provider to your project.