You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/external/togetherai.mdx
+3-3
Original file line number
Diff line number
Diff line change
@@ -18,7 +18,7 @@ Start by creating a Helicone account and obtaining your API key from the [Helico
18
18
19
19
### 2. Implement Helicone
20
20
21
-
Wherever you use the TogetherAI API, simply substitute the base URL with `https://together.hconeai.com` and include the `Helicone-Auth` header with your Helicone API key.
21
+
Wherever you use the TogetherAI API, simply substitute the base URL with `https://together.helicone.ai` and include the `Helicone-Auth` header with your Helicone API key.
base_url='https://together.hconeai.com/v1', # Change to Helicone
30
+
base_url='https://together.helicone.ai/v1', # Change to Helicone
31
31
default_headers={
32
32
"Helicone-Auth": f"Bearer {HELICONE_API_KEY}", # Add Helicone API Key
33
33
}
@@ -36,4 +36,4 @@ client = OpenAI(
36
36
37
37
### 3. Start Utilizing Helicone
38
38
39
-
Congratulations! You are now using Helicone to proxy your requests to TogetherAI. You can now leverage Helicone's features such as data ingestion, caching, and analytics.
39
+
Congratulations! You are now using Helicone to proxy your requests to TogetherAI. You can now leverage Helicone's features such as data ingestion, caching, and analytics.
<Info>**Who can use this feature**: Anyone on any [plan](https://www.helicone.ai/pricing). However, the maximum number of caches you can store within a bucket is `20`. If you need to store more, you will need to upgrade to an enterprise plan.</Info>
8
+
<Info>
9
+
**Who can use this feature**: Anyone on any
10
+
[plan](https://www.helicone.ai/pricing). However, the maximum number of caches
11
+
you can store within a bucket is `20`. If you need to store more, you will
12
+
need to upgrade to an enterprise plan.
13
+
</Info>
9
14
10
15
## Introduction
11
-
Caching, by temporarily storing data closer to the user at the edge, can significantly speed up access time and enhance your application performance.
12
16
13
-
Helicone uses [Cloudflare Workers](https://www.cloudflare.com/network/) to temporarily store data closer to the user to ensure low latency, resulting in faster responses and an efficient app development process.
17
+
Caching, by temporarily storing data closer to the user at the edge, can significantly speed up access time and enhance your application performance.
18
+
19
+
Helicone uses [Cloudflare Workers](https://www.cloudflare.com/network/) to temporarily store data closer to the user to ensure low latency, resulting in faster responses and an efficient app development process.
14
20
15
21
<Framecaption="Dashboard view of cache hits, cost and time saved. ">
16
22
<imgsrc="/images/example-cache.png" />
17
23
</Frame>
18
24
19
-
20
25
## Why Cache
26
+
21
27
- Faster response for commonly asked questions, resulting in better experience for your users.
22
28
- Lower latency and reduce the load on backend resources by pre-computing results or frequently accessed data, so you can develop your app more efficiently.
23
29
- Save money while testing your app by making fewer calls to model providers such as OpenAI.
24
30
- Determine the most common requests with your application and visualize on a dashboard.
25
31
26
32
## Quick Start
33
+
27
34
To get started, set `Helicone-Cache-Enabled` to true in the headers, or use the Python or NPM packages to turn it on via parameters.
28
35
29
36
<CodeGroup>
30
37
31
38
```bash Curl
32
-
curl https://oai.hconeai.com/v1/completions \
39
+
curl https://oai.helicone.ai/v1/completions \
33
40
-H 'Content-Type: application/json' \
34
41
-H 'Authorization: Bearer YOUR_API_KEY' \
35
42
-H 'Helicone-Cache-Enabled: true'\ # add this header and set to true
|`Helicone-Cache-Enabled`(required) | Set to `true` to enable storing and loading from your cache. |
94
-
|`Cache-Control` (optional) | Configure cache limit as a `string` based on the [Cloudflare Cache Directive](https://developers.cloudflare.com/cache/about/cache-control#cache-control-directives). Currently we only support `max-age`, but we will be adding more configuration options soon. I.e. 1 hour is `max-age=3600`.|
95
-
|`Helicone-Cache-Bucket-Max-Size` (optional) | Configure your Cache Bucket size as a `number`. |
96
-
|`Helicone-Cache-Seed` (optional) | Define a separate cache state as a `string` to generate predictable results, i.e. `user-123`.|
|`Helicone-Cache-Enabled` (required) | Set to `true` to enable storing and loading from your cache.|
102
+
|`Cache-Control` (optional) | Configure cache limit as a `string` based on the [Cloudflare Cache Directive](https://developers.cloudflare.com/cache/about/cache-control#cache-control-directives). Currently we only support `max-age`, but we will be adding more configuration options soon. I.e. 1 hour is `max-age=3600`.|
103
+
|`Helicone-Cache-Bucket-Max-Size` (optional) | Configure your Cache Bucket size as a `number`. |
104
+
|`Helicone-Cache-Seed` (optional) | Define a separate cache state as a `string` to generate predictable results, i.e. `user-123`.|
97
105
98
-
<Info>Header values have to be strings. For example, `"Helicone-Cache-Bucket-Max-Size": "10"`. </Info>
106
+
<Info>
107
+
Header values have to be strings. For example,
108
+
`"Helicone-Cache-Bucket-Max-Size": "10"`.{""}
109
+
</Info>
99
110
100
111
### Changing Cache Limit
101
-
The default cache limit is 7 days. To change the limit, add the `Cache-Control` header to your request.
112
+
113
+
The default cache limit is 7 days. To change the limit, add the `Cache-Control` header to your request.
102
114
103
115
**Example**: Setting the cache limit to 30 days, aka `2592000 seconds`
104
116
105
117
```tsx
106
118
"Cache-Control": "max-age=2592000"
107
119
```
108
-
<Note>The max cache limit is 365 days, or `max-age=31536000`. </Note>
109
-
110
120
121
+
<Note>The max cache limit is 365 days, or `max-age=31536000`. </Note>
111
122
112
123
### Configuring Bucket Size
113
124
@@ -116,14 +127,14 @@ Simply add `Helicone-Cache-Bucket-Max-Size` with some number to choose how large
116
127
<CodeGroup>
117
128
118
129
```python Python
119
-
openai.api_base ="https://oai.hconeai.com/v1"
130
+
openai.api_base ="https://oai.helicone.ai/v1"
120
131
121
132
openai.Completion.create(
122
133
model="text-davinci-003",
123
134
prompt="Say this is a test",
124
135
headers={
125
136
"Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
126
-
"Helicone-Cache-Enabled": "true", # mandatory
137
+
"Helicone-Cache-Enabled": "true", # mandatory
127
138
"Helicone-Cache-Bucket-Max-Size": "3", # set cache bucket size to 3
128
139
}
129
140
)
@@ -133,9 +144,9 @@ headers={
133
144
import { Configuration, OpenAIApi } from"openai";
134
145
const configuration =newConfiguration({
135
146
apiKey: process.env.OPENAI_API_KEY,
136
-
basePath: "https://oai.hconeai.com/v1",
147
+
basePath: "https://oai.helicone.ai/v1",
137
148
defaultHeaders: {
138
-
"Helicone-Cache-Enabled": "true", // mandatory
149
+
"Helicone-Cache-Enabled": "true", // mandatory
139
150
"Helicone-Cache-Bucket-Max-Size": "3", // set cache bucket size to 3
140
151
},
141
152
});
@@ -146,7 +157,6 @@ const openai = new OpenAIApi(configuration);
146
157
147
158
**Example**: A bucket size of 3
148
159
149
-
150
160
```
151
161
openai.completion("give me a random number") -> "42"
152
162
# Cache Miss
@@ -158,11 +168,16 @@ openai.completion("give me a random number") -> "17"
158
168
openai.completion("give me a random number") -> This will randomly choose 42 | 47 | 17
159
169
# Cache Hit
160
170
```
161
-
<Note>The max number of caches you can store is `20` within a bucket, if you want more you will need to upgrade to an enterprise [plan](https://www.helicone.ai/pricing).</Note>
171
+
172
+
<Note>
173
+
The max number of caches you can store is `20` within a bucket, if you want
174
+
more you will need to upgrade to an enterprise
175
+
[plan](https://www.helicone.ai/pricing).
176
+
</Note>
162
177
163
178
### Adding Cache Seed
164
-
When you make a request to Helicone with the same seed, you will receive the same cached response for the same query. This feature allows for predictable results, which can be beneficial in scenarios where you want to have a consistent cache across multiple requests.
165
179
180
+
When you make a request to Helicone with the same seed, you will receive the same cached response for the same query. This feature allows for predictable results, which can be beneficial in scenarios where you want to have a consistent cache across multiple requests.
166
181
167
182
To set a cache seed, add a header called `Helicone-Cache-Seed` with a string value for the seed.
168
183
@@ -190,15 +205,19 @@ openai.completion("give me a random number") -> "42"
190
205
# Cache Seed "user-456"
191
206
openai.completion("give me a random number") -> "17"
192
207
```
193
-
<Tip> If you don’t like one of generated response stored in cache, you can update your seed to a different value as a way to clear your cache. </Tip>
208
+
209
+
<Tip>
210
+
{""}
211
+
If you don’t like one of generated response stored in cache, you can update your
212
+
seed to a different value as a way to clear your cache.{""}
213
+
</Tip>{""}
194
214
195
215
### Extracting Cache Response Headers
196
216
197
217
When cache is enabled, you can capture the cache status from the headers of the response, such as a `cache hit / miss` and the `cache bucket index` of the response returned.
198
218
199
-
200
219
```ts
201
-
helicone-cache: "HIT"|"MISS"// indicates whether the response was cached.
220
+
helicone-cache: "HIT"|"MISS"// indicates whether the response was cached.
202
221
helicone-cache-bucket-idx: number// indicates the cache bucket index used.
203
222
```
204
223
@@ -207,7 +226,7 @@ helicone-cache-bucket-idx: number // indicates the cache bucket index used.
0 commit comments