Heroku AI SDK Provider

A TypeScript provider for the Vercel AI SDK that enables you to use Heroku's AI inference capabilities in your applications. This provider supports both chat completions and embeddings through Heroku's AI infrastructure.

Features

🤖 Chat Completions: Support for Claude 4.5 Sonnet, Claude 4 Sonnet, Claude 3.7 Sonnet, Claude 3.5 Haiku, gpt-oss-120b, Nova Lite, Nova Pro and other Heroku-hosted models
🧠 Embeddings: Generate vector embeddings using Cohere's multilingual embedding model
🖼️ Image Generation: Create images with Stability AI’s Stable Image Ultra diffusion model
🔧 Tool Support: Function calling capabilities for building AI agents and assistants
📡 Streaming: Real-time streaming responses for chat completions
🔐 TypeScript: Full TypeScript support with comprehensive type definitions
⚡ Dual Module Support: Compatible with both ESM and CommonJS projects

Installation

npm install heroku-ai-provider
# or
yarn add heroku-ai-provider
# or
pnpm add heroku-ai-provider

Prerequisites

Before using this provider, you'll need:

A Heroku account with access to AI services
Heroku AI API keys for inference and/or embeddings
The Vercel AI SDK installed in your project

npm install ai

Basic Setup

Environment Variables

Set your Heroku AI API keys as environment variables:

# For chat completions
INFERENCE_KEY=your_inference_api_key

# For embeddings
EMBEDDING_KEY=your_embedding_api_key

# For image generations
DIFFUSION_KEY=your_diffusion_api_key

# Optional: Custom API endpoints
INFERENCE_URL=https://us.inference.heroku.com
EMBEDDING_URL=https://us.inference.heroku.com
DIFFUSION_URL=https://us.inference.heroku.com

Basic Configuration

import { heroku } from "heroku-ai-provider";

const model = heroku.chat("claude-4-sonnet");

Custom Configuration

import { createHerokuAI } from "heroku-ai-provider";

const client = createHerokuAI({
  chatApiKey: "your_inference_api_key",
  embeddingsApiKey: "your_embedding_api_key",
  chatBaseUrl: "https://us.inference.heroku.com/v1/chat/completions",
  embeddingsBaseUrl: "https://us.inference.heroku.com/v1/embeddings",
  imageApiKey: "your_diffusion_api_key",
  imageBaseUrl: "https://us.inference.heroku.com/v1/images/generations",
});

Usage Examples

Chat Completions

Basic Chat

import { generateText } from "ai";
import { heroku } from "heroku-ai-provider";

const { text } = await generateText({
  model: heroku.chat("claude-4-sonnet"),
  prompt: "What is the capital of France?",
});

console.log(text); // "The capital of France is Paris."

Streaming Chat

import { streamText, stepCountIs } from "ai";
import { heroku } from "heroku-ai-provider";

const { textStream } = await streamText({
  model: heroku.chat("claude-3-haiku"),
  prompt: "Write a short story about a robot learning to paint.",
});

for await (const delta of textStream) {
  process.stdout.write(delta);
}

Chat with System Message

import { generateText } from "ai";
import { heroku } from "heroku-ai-provider";

const { text } = await generateText({
  model: heroku.chat("claude-4-sonnet"),
  system: "You are a helpful assistant that explains complex topics simply.",
  prompt: "Explain quantum computing",
});

Tool/Function Calling

import { generateText, tool, stepCountIs } from "ai";
import { heroku } from "heroku-ai-provider";
import { z } from "zod";

const { text } = await generateText({
  model: heroku.chat("claude-4-sonnet"),
  prompt: "What is the weather like in New York?",
  tools: {
    getWeather: tool({
      description: "Get the current weather for a location",
      parameters: z.object({
        location: z.string().describe("The city and state"),
      }),
      execute: async ({ location }) => {
        // Simulate weather API call
        return {
          location,
          temperature: 72,
          condition: "sunny",
        };
      },
    }),
  },
  stopWhen: stepCountIs(5),
});

Advanced Tool Usage with Multiple Steps

import { generateText, tool, stepCountIs } from "ai";
import { heroku } from "heroku-ai-provider";
import { z } from "zod";

const { text, steps } = await generateText({
  model: heroku.chat("claude-4-sonnet"),
  prompt:
    "Check the weather in New York and then suggest appropriate clothing.",
  tools: {
    getWeather: tool({
      description: "Get the current weather for a location",
      parameters: z.object({
        location: z.string().describe("The city and state"),
      }),
      execute: async ({ location }) => {
        return {
          location,
          temperature: 45,
          condition: "rainy",
          humidity: 80,
        };
      },
    }),
    suggestClothing: tool({
      description: "Suggest appropriate clothing based on weather conditions",
      inputSchema: z.object({
        temperature: z.number().describe("Temperature in Fahrenheit"),
        condition: z.string().describe("Weather condition"),
        humidity: z.number().optional().describe("Humidity percentage"),
      }),
      execute: async ({ temperature, condition, humidity }) => {
        return {
          suggestions: [
            "Waterproof jacket",
            "Warm layers",
            "Waterproof shoes",
            "Umbrella",
          ],
          reasoning: `Given ${temperature}°F and ${condition} weather${humidity ? ` with ${humidity}% humidity` : ""}, you'll want to stay warm and dry.`,
        };
      },
    }),
  },
  stopWhen: stepCountIs(5),
});

console.log("Final response:", text);
console.log("Tool execution steps:", steps.length);

Embeddings

Basic Embeddings

import { embed } from "ai";
import { heroku } from "heroku-ai-provider";

const { embedding } = await embed({
  model: heroku.embedding("cohere-embed-multilingual"),
  value: "Hello, world!",
});

console.log(embedding); // [0.1, 0.2, -0.3, ...]

Multiple Embeddings

import { embedMany } from "ai";
import { heroku } from "heroku-ai-provider";

const { embeddings } = await embedMany({
  model: heroku.embedding("cohere-embed-multilingual"),
  values: ["First document", "Second document", "Third document"],
});

console.log(embeddings.length); // 3

Using the Convenience Function

import { createEmbedFunction } from "heroku-ai-provider";

// Create a reusable embed function
const embedText = createEmbedFunction({
  apiKey: process.env.EMBEDDING_KEY!,
  model: "cohere-embed-multilingual",
});

const embedding = await embedText("Hello, world!");
console.log(embedding); // [0.1, 0.2, -0.3, ...]

Image Generation

import { experimental_generateImage as generateImage } from "ai";
import { heroku } from "heroku-ai-provider";

const result = await generateImage({
  model: heroku.image("stable-image-ultra"),
  prompt: "A watercolor illustration of a lighthouse at sunrise",
  size: "1024x1024",
});

const imageBytes = result.image.uint8Array;
console.log("Generated image bytes length:", imageBytes.length);

Configuration Options

HerokuProviderSettings

interface HerokuProviderSettings {
  // API keys (falls back to environment variables)
  chatApiKey?: string; // INFERENCE_KEY
  embeddingsApiKey?: string; // EMBEDDING_KEY
  imageApiKey?: string; // DIFFUSION_KEY

  // Base URLs (falls back to environment variables or defaults)
  chatBaseUrl?: string; // INFERENCE_URL
  embeddingsBaseUrl?: string; // EMBEDDING_URL
  imageBaseUrl?: string; // DIFFUSION_URL
}

Supported Models

Chat Models

claude-4-sonnet - Claude 4 Sonnet by Anthropic
claude-4-5-sonnet - Claude 4.5 Sonnet by Anthropic
claude-3-haiku - Claude 3 Haiku by Anthropic
claude-3-7-sonnet - Claude 3.7 Sonnet by Anthropic
claude-3-5-haiku - Claude 3.5 Haiku by Anthropic
claude-3-5-sonnet-latest - Claude 3.5 Sonnet by Anthropic
gpt-oss-120b - gpt-oss-120b by OpenAI
nova-lite - Nova Lite by Amazon
nova-pro - Nova Pro by Amazon

Embedding Models

cohere-embed-multilingual - Multilingual embedding model by Cohere

Image Models

stable-image-ultra - Stable Image Ultra diffusion model by Stability AI

Framework Integration

Next.js App Router

// app/api/chat/route.ts
import { streamText, stepCountIs } from "ai";
import { heroku } from "heroku-ai-provider";

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: heroku.chat("claude-4-sonnet"),
    messages,
    stopWhen: stepCountIs(5), // Enable multi-step tool conversations
  });

  return result.toDataStreamResponse();
}

Next.js with Tool Support

// app/api/chat/route.ts
import { streamText, tool } from "ai";
import { heroku } from "heroku-ai-provider";
import { z } from "zod";

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: heroku.chat("claude-4-sonnet"),
    messages,
    tools: {
      getTime: tool({
        description: "Get the current time",
        inputSchema: z.object({
          timezone: z
            .string()
            .optional()
            .describe("Timezone (e.g., 'America/New_York')"),
        }),
        execute: async ({ timezone = "UTC" }) => {
          return {
            time: new Date().toLocaleString("en-US", { timeZone: timezone }),
            timezone,
          };
        },
      }),
    },
    stopWhen: stepCountIs(5),
  });

  return result.toDataStreamResponse();
}

Express.js

import express from "express";
import { generateText } from "ai";
import { heroku } from "heroku-ai-provider";

const app = express();

app.post("/chat", async (req, res) => {
  const { prompt } = req.body;

  const { text } = await generateText({
    model: heroku.chat("claude-3-haiku"),
    prompt,
  });

  res.json({ response: text });
});

Error Handling

The provider includes comprehensive error handling with user-friendly error messages:

import {
  createHerokuAI,
  isConfigurationError,
  isTemporaryServiceError,
} from "heroku-ai-provider";

try {
  const result = await generateText({
    model: heroku.chat("claude-4-sonnet"),
    prompt: "Hello!",
  });
} catch (error) {
  if (isConfigurationError(error)) {
    console.error("Configuration error:", error.message);
    // Handle API key or URL configuration issues
  } else if (isTemporaryServiceError(error)) {
    console.error("Service error:", error.message);
    // Handle temporary service issues (retry logic)
  } else {
    console.error("Unexpected error:", error);
  }
}

Troubleshooting

Common Issues

Authentication Errors

Issue: "Chat API key is required" or "Embeddings API key is required"
Solution: Ensure your API keys are set in environment variables or passed directly to createHerokuAI()

Model Not Found

Issue: "Unsupported chat model" or "Unsupported embedding model"
Solution: Check that you're using a supported model from the list above

Network Errors

Issue: Connection timeouts or network failures
Solution: Verify your internet connection and that Heroku's AI services are accessible

URL Configuration

Issue: "Invalid URL format" errors
Solution: Ensure custom URLs are valid and use HTTP/HTTPS protocol

Tool Execution Issues

Issue: Tools are called but AI doesn't provide final response
Solution: Ensure you configure stopWhen (for example, stopWhen: stepCountIs(5)) so the model can complete multi-step tool conversations

Schema Validation Errors

Issue: "Unrecognized request argument" errors when using tools
Solution: This provider automatically filters out problematic schema properties (like $schema) that some validation libraries add

Contributing

We welcome contributions! Please follow these steps:

Fork the repository
Create a feature branch: git checkout -b feature/amazing-feature
Make your changes and add tests
Ensure tests pass: npm test
Lint your code: npm run lint
Commit your changes: git commit -m 'Add amazing feature'
Push to the branch: git push origin feature/amazing-feature
Open a Pull Request

Development Setup

# Clone the repository
git clone https://github.com/julianduque/heroku-ai-provider.git
cd heroku-ai-provider

# Install dependencies
pnpm install

# Run tests
pnpm test

# Build the project
pnpm build

# Lint code
pnpm lint

Testing

The project uses Jest for testing. Run tests with:

# Run all tests
pnpm test

# Run tests in watch mode
pnpm test --watch

# Run tests with coverage
pnpm test --coverage

License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.

Resources

📖 Documentation: Heroku AI Documentation
🐛 Issues: GitHub Issues

Related Projects

Vercel AI SDK - The AI SDK this provider integrates with
Heroku CLI - Command line interface for Heroku
Heroku AI Services - Official Heroku AI documentation

Name		Name	Last commit message	Last commit date
Latest commit History 24 Commits
.github/workflows		.github/workflows
.husky		.husky
dist		dist
docs		docs
examples		examples
scripts		scripts
src		src
tests		tests
.gitignore		.gitignore
.npmignore		.npmignore
.prettierignore		.prettierignore
AGENTS.md		AGENTS.md
LICENSE		LICENSE
README.md		README.md
commitlint.config.cjs		commitlint.config.cjs
eslint.config.mjs		eslint.config.mjs
jest.config.cjs		jest.config.cjs
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
tsconfig.cjs.json		tsconfig.cjs.json
tsconfig.esm.json		tsconfig.esm.json
tsconfig.json		tsconfig.json
typedoc.json		typedoc.json

License

julianduque/heroku-ai-provider

Folders and files

Latest commit

History

Repository files navigation

Heroku AI SDK Provider

Features

Installation

Prerequisites

Basic Setup

Environment Variables

Basic Configuration

Custom Configuration

Usage Examples

Chat Completions

Basic Chat

Streaming Chat

Chat with System Message

Tool/Function Calling

Advanced Tool Usage with Multiple Steps

Embeddings

Basic Embeddings

Multiple Embeddings

Using the Convenience Function

Image Generation

Configuration Options

HerokuProviderSettings

Supported Models

Chat Models

Embedding Models

Image Models

Framework Integration

Next.js App Router

Next.js with Tool Support

Express.js

Error Handling

Troubleshooting

Common Issues

Authentication Errors

Model Not Found

Network Errors

URL Configuration

Tool Execution Issues

Schema Validation Errors

Contributing

Development Setup

Testing

License

Resources

Related Projects

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages