Update OpenAI for new ChatGPT-3.5 fine-tuning release

docs/guides/integrations/other/openai-api.md

@@ -0,0 +1,68 @@
+---
+slug: /guides/integrations/openai-api
+description: How to use W&B with the OpenAI API.
+displayed_sidebar: default
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# OpenAI API
+
+Use the Weights & Biases OpenAI API integration to log requests, responses, token counts and model metadata with 1 line of code for all OpenAI models, including fine-tuned models.
+
+**[Try in a Colab Notebook here →](https://github.com/wandb/examples/blob/master/colabs/openai/OpenAI_API_Autologger_Quickstart.ipynb)**
+
+With just 1 line of code you can now automatically log inputs and outputs from the OpenAI Python SDK to Weights & Biases! 
+
+![](/images/integrations/open_ai_autolog.png)
+
+Once you start logging your API inputs and outputs you can quickly evaluate the performance of difference prompts, compare different model settings (such as temperature), and track other usage metrics such as token usage.
+
+To get started, pip install the `wandb` library, then follow the steps below:
+
+### 1. Import autolog and initialise it
+First, import `autolog` from `wandb.integration.openai` and initialise it.  
+
+```python
+import os
+import openai
+from wandb.integration.openai import autolog
+
+autolog({"project":"gpt5"})
+```
+
+You can optionally pass a dictionary with argument that `wandb.init()` accepts to `autolog`. This includes a project name, team name, entity, and more. For more information about [`wandb.init`](../../../ref/python/init.md), see the API Reference Guide.
+
+### 2. Call the OpenAI API
+Each call you make to the OpenAI API will now be logged to Weights & Biases automatically.
+
+```python
+os.environ["OPENAI_API_KEY"] = "XXX"
+
+chat_request_kwargs = dict(
+    model="gpt-3.5-turbo",
+    messages=[
+        {"role": "system", "content": "You are a helpful assistant."},
+        {"role": "user", "content": "Who won the world series in 2020?"},
+        {"role": "assistant", "content": "The Los Angeles Dodgers"},
+        {"role": "user", "content": "Where was it played?"},
+    ],
+)
+response = openai.ChatCompletion.create(**chat_request_kwargs)
+```
+
+### 3. View your OpenAI API inputs and responses
+
+Click on the Weights & Biases [run](../../runs/intro.md) link generated by `autolog` in **step 1**. This will redirect you to your project workspace in the W&B App.
+
+Select a run you created to view the trace table, trace timeline and the model architecture of the OpenAI LLM used.
+
+### 4. Disable autolog
+We recommend that you call `disable()` to close all W&B processes when you are finished using the OpenAI API.
+
+```python
+autolog.disable()
+```
+
+Now your inputs and completions will be logged to Weights & Biases, ready for analysis or to be shared with colleagues.

docs/guides/integrations/other/openai-fine-tuning.md

@@ -0,0 +1,212 @@
+---
+slug: /guides/integrations/openai
+description: How to Fine-Tune OpenAI models using W&B.
+displayed_sidebar: default
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# OpenAI Fine-Tuning
+
+With Weights & Biases you can log your OpenAI ChatGPT-3.5 or GPT-4 model's fine-tuning metrics and configuration to Weights & Biases to analyse and understand the performance of your newly fine-tuned models and share the results with your colleagues.
+
+## Sync your OpenAI Fine-Tuning Results in 1 Line
+
+If you use OpenAI's API to [fine-tune OpenAI models](https://platform.openai.com/docs/guides/fine-tuning/), you can now use the W&B integration to track experiments, models, and datasets in your central dashboard.
+
+<Tabs
+  defaultValue="cli"
+  values={[
+    {label: 'Command Line', value: 'cli'},
+    {label: 'Python', value: 'python_sdk'},
+  ]}>
+  <TabItem value="cli">
+
+```shell-session
+openai wandb sync
+```
+
+  </TabItem>
+  <TabItem value="python_sdk">
+
+```python
+from openai.wandb_logger import WandbLogger
+
+WandbLogger.sync(project="OpenAI-Fine-Tune")
+```
+  </TabItem>
+</Tabs>
+
+<!-- ![](/images/integrations/open_ai_api.png) -->
+![](/images/integrations/open_ai_auto_scan.png)
+
+### Check out interactive examples
+
+* [Demo Colab](http://wandb.me/openai-colab)
+* [Report - OpenAI Fine-Tuning Exploration and Tips](http://wandb.me/openai-report)
+
+### Sync your fine-tunes with one line
+
+Make sure you are using latest version of openai and wandb.
+
+```shell-session
+pip install --upgrade openai wandb
+```
+
+Then sync your results from the command line or from your script.
+
+<Tabs
+  defaultValue="cli"
+  values={[
+    {label: 'Command Line', value: 'cli'},
+    {label: 'Python', value: 'python_sdk'},
+  ]}>
+  <TabItem value="cli">
+
+```shell-session
+# one line command
+openai wandb sync
+
+# passing optional parameters
+openai wandb sync --help
+```
+  </TabItem>
+  <TabItem value="python_sdk">
+
+```python
+from openai.wandb_logger import WandbLogger
+
+# one line command
+WandbLogger.sync()
+
+# passing optional parameters
+WandbLogger.sync(
+    id=None,
+    n_fine_tunes=None,
+    project="OpenAI-Fine-Tune",
+    entity=None,
+    force=False,
+    **kwargs_wandb_init
+)
+```
+  </TabItem>
+</Tabs>
+
+When you sync your results, wandb checks OpenAI for newly completed fine-tunes and automatically adds them to your dashboard.
+
+### Optional arguments
+
+| Argument                 | Description                                                                                                               |
+| ------------------------ | ------------------------------------------------------------------------------------------------------------------------- |
+| -i ID, --id ID           | The id of the fine-tune (optional)                                                                                        |
+| -n N, --n\_fine\_tunes N | Number of most recent fine-tunes to log when an id is not provided. By default, every fine-tune is synced.                |
+| --project PROJECT        | Name of the Weights & Biases project where you're sending runs. By default, it is "OpenAI-Fine-Tune".                                                 |
+| --entity ENTITY          | Weights & Biases Username or team name where you're sending runs. By default, your default entity is used, which is usually your username. |
+| --force                  | Forces logging and overwrite existing wandb run of the same fine-tune.                                                    |
+| --legacy                  | Log results from the legacy OpenAI GPT-3 fine-tune api.                                                    |
+| \*\*kwargs\_wandb\_init  | In python, any additional argument is directly passed to [`wandb.init()`](../../../ref/python/init.md)                    |
+
+### Inspect sample predictions
+
+Use [Tables](../../tables/intro.md) to better visualize sample predictions and compare models.
+
+![](/images/integrations/open_ai_inspect_sample.png)
+
+Create a new run:
+
+```python
+run = wandb.init(project="OpenAI-Fine-Tune", job_type="eval")
+```
+
+Retrieve a model id for inference.
+
+You can use automatically logged artifacts to retrieve your latest model:
+
+```python
+ft_artifact = run.use_artifact("ENTITY/PROJECT/fine_tune_details:latest")
+fine_tuned_model = ft_artifact.metadata["fine_tuned_model"]
+```
+
+You can also retrieve your validation file:
+
+```python
+artifact_valid = run.use_artifact("ENTITY/PROJECT/FILENAME:latest")
+valid_file = artifact_valid.get_path("FILENAME").download()
+```
+
+Perform some inferences using OpenAI API:
+
+```python
+# perform inference and record results
+my_prompts = ["PROMPT_1", "PROMPT_2"]
+results = []
+for prompt in my_prompts:
+    res = openai.ChatCompletion.create(model=fine_tuned_model,
+                                   prompt=prompt,
+                                   ...)
+    results.append(res["choices"][0]["text"])
+```
+
+Log your results with a Table:
+
+```python
+table = wandb.Table(columns=['prompt', 'completion'],
+                    data=list(zip(my_prompts, results)))
+```
+
+## Frequently Asked Questions
+
+### How do I share my fine-tune resutls with my team in W&B?
+
+Sync all your runs to your team account with:
+
+```shell-session
+openai wandb sync --entity MY_TEAM_ACCOUNT
+```
+
+### How can I organize my runs?
+
+Your W&B runs are automatically organized and can be filtered/sorted based on any configuration parameter such as job type, base model, learning rate, training filename and any other hyper-parameter.
+
+In addition, you can rename your runs, add notes or create tags to group them.
+
+Once you’re satisfied, you can save your workspace and use it to create report, importing data from your runs and saved artifacts (training/validation files).
+
+### How can I access my fine-tune details?
+
+Fine-tune details are logged to W&B as artifacts and can be accessed with:
+
+```python
+import wandb
+
+ft_artifact = wandb.run.use_artifact('USERNAME/PROJECT/job_details:VERSION')
+```
+
+where `VERSION` is either:
+
+* a version number such as `v2`
+* the fine-tune id such as `ft-xxxxxxxxx`
+* an alias added automatically such as `latest` or manually
+
+You can then access fine-tune details through `artifact_job.metadata`. For example, the fine-tuned model can be retrieved with `artifact_job.metadata[`"`fine_tuned_model"]`.
+
+### What if a fine-tune was not synced successfully?
+
+You can always call again `openai wandb sync` and we will re-sync any run that was not synced successfully.
+
+If needed, you can call `openai wandb sync --id fine_tune_id --force` to force re-syncing a specific fine-tune.
+
+### Can I track my datasets with W&B?
+
+Yes, you can integrate your entire pipeline to W&B through Artifacts, including creating your dataset, splitting it, training your models and evaluating them!
+
+This will allow complete traceability of your models.
+
+![](/images/integrations/open_ai_faq_can_track.png)
+
+## Resources
+
+* [OpenAI Fine-tuning Documentation](https://platform.openai.com/docs/guides/fine-tuning/) is very thorough and contains many useful tips
+* [Demo Colab](http://wandb.me/openai-colab)
+* [Report - OpenAI Fine-Tuning Exploration & Tips](http://wandb.me/openai-report)