Skip to content

Commit c7b3f0b

Browse files
authored
feat (docs): Add SambaNova community provider (#5281)
1 parent 19b816c commit c7b3f0b

File tree

1 file changed

+175
-0
lines changed

1 file changed

+175
-0
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
---
2+
title: SambaNova
3+
description: Learn how to use the SambaNova provider for the AI SDK.
4+
---
5+
6+
# SambaNova Provider
7+
8+
[sambanova-ai-provider](https://github.com/sambanova/sambanova-ai-provider) contains language model support for the SambaNova API.
9+
10+
API keys can be obtained from the [SambaNova Cloud Platform](https://cloud.sambanova.ai/apis).
11+
12+
## Setup
13+
14+
The SambaNova provider is available via the `sambanova-ai-provider` module. You can install it with:
15+
16+
<Tabs items={['pnpm', 'npm', 'yarn']}>
17+
<Tab>
18+
<Snippet text="pnpm add sambanova-ai-provider" dark />
19+
</Tab>
20+
<Tab>
21+
<Snippet text="npm install sambanova-ai-provider" dark />
22+
</Tab>
23+
<Tab>
24+
<Snippet text="yarn add sambanova-ai-provider" dark />
25+
</Tab>
26+
</Tabs>
27+
28+
### Environment variables
29+
30+
Create a `.env` file with a `SAMBANOVA_API_KEY` variable.
31+
32+
## Provider Instance
33+
34+
You can import the default provider instance `sambanova` from `sambanova-ai-provider`:
35+
36+
```ts
37+
import { sambanova } from 'sambanova-ai-provider';
38+
```
39+
40+
If you need a customized setup, you can import `createSambaNova` from `sambanova-ai-provider` and create a provider instance with your settings:
41+
42+
```ts
43+
import { createSambaNova } from 'sambanova-ai-provider';
44+
45+
const sambanova = createSambaNova({
46+
// Optional settings
47+
});
48+
```
49+
50+
You can use the following optional settings to customize the SambaNova provider instance:
51+
52+
- **baseURL** _string_
53+
54+
Use a different URL prefix for API calls, e.g. to use proxy servers.
55+
The default prefix is `https://api.sambanova.ai/v1`.
56+
57+
- **apiKey** _string_
58+
59+
API key that is being sent using the `Authorization` header.
60+
It defaults to the `SAMBANOVA_API_KEY` environment variable.
61+
62+
- **headers** _Record&lt;string,string&gt;_
63+
64+
Custom headers to include in the requests.
65+
66+
- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_
67+
68+
Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.
69+
Defaults to the global `fetch` function.
70+
You can use it as a middleware to intercept requests,
71+
or to provide a custom fetch implementation for e.g. testing.
72+
73+
## Models
74+
75+
You can use [SambaNova models](https://docs.sambanova.ai/cloud/docs/get-started/supported-models) on a provider instance.
76+
The first argument is the model id, e.g. `Meta-Llama-3.1-70B-Instruct`.
77+
78+
```ts
79+
const model = sambanova('Meta-Llama-3.1-70B-Instruct');
80+
```
81+
82+
## Tested models and capabilities
83+
84+
This provider is capable of generating and streaming text, and interpreting image inputs.
85+
86+
At least it has been tested with the following features (which use the `/chat/completion` endpoint):
87+
88+
| Chat completion | Image input |
89+
| ------------------- | ------------------- |
90+
| <Check size={18} /> | <Check size={18} /> |
91+
92+
### Image input
93+
94+
You need to use any of the following models for visual understanding:
95+
96+
- Llama3.2-11B-Vision-Instruct
97+
- Llama3.2-90B-Vision-Instruct
98+
99+
SambaNova does not support URLs, but the ai-sdk is able to download the file and send it to the model.
100+
101+
## Example Usage
102+
103+
Basic demonstration of text generation using the SambaNova provider.
104+
105+
```ts
106+
import { createSambaNova } from 'sambanova-ai-provider';
107+
import { generateText } from 'ai';
108+
109+
const sambanova = createSambaNova({
110+
apiKey: 'YOUR_API_KEY',
111+
});
112+
113+
const model = sambanova('Meta-Llama-3.1-70B-Instruct');
114+
115+
const { text } = await generateText({
116+
model,
117+
prompt: 'Hello, nice to meet you.',
118+
});
119+
120+
console.log(text);
121+
```
122+
123+
You will get an output text similar to this one:
124+
125+
```
126+
Hello. Nice to meet you too. Is there something I can help you with or would you like to chat?
127+
```
128+
129+
## Intercepting Fetch Requests
130+
131+
This provider supports [Intercepting Fetch Requests](https://sdk.vercel.ai/examples/providers/intercepting-fetch-requests).
132+
133+
### Example
134+
135+
```ts
136+
import { createSambaNova } from 'sambanova-ai-provider';
137+
import { generateText } from 'ai';
138+
139+
const sambanovaProvider = createSambaNova({
140+
apiKey: 'YOUR_API_KEY',
141+
fetch: async (url, options) => {
142+
console.log('URL', url);
143+
console.log('Headers', JSON.stringify(options.headers, null, 2));
144+
console.log(`Body ${JSON.stringify(JSON.parse(options.body), null, 2)}`);
145+
return await fetch(url, options);
146+
},
147+
});
148+
149+
const model = sambanovaProvider('Meta-Llama-3.1-70B-Instruct');
150+
151+
const { text } = await generateText({
152+
model,
153+
prompt: 'Hello, nice to meet you.',
154+
});
155+
```
156+
157+
And you will get an output like this:
158+
159+
```bash
160+
URL https://api.sambanova.ai/v1/chat/completions
161+
Headers {
162+
"Content-Type": "application/json",
163+
"Authorization": "Bearer YOUR_API_KEY"
164+
}
165+
Body {
166+
"model": "Meta-Llama-3.1-70B-Instruct",
167+
"temperature": 0,
168+
"messages": [
169+
{
170+
"role": "user",
171+
"content": "Hello, nice to meet you."
172+
}
173+
]
174+
}
175+
```

0 commit comments

Comments
 (0)