feat: openai wandb sync (#64)

* feat: log fine_tune with wandb

* feat: ensure we are logged in

* feat: cli wandb namespace

* feat: add fine_tuned_model to summary

* feat: log training & validation files

* feat: re-log if was not successful or force

* doc: add docstring

* feat: set wandb api only when needed

* fix: train/validation files are inputs

* feat: rename artifact type

* feat: improve config logging

* feat: log all jobs by default

* feat: log job details

* feat: log -> sync

* feat: cli wandb log -> sync

* fix: validation_files not always present

* feat: format created_at + style

* feat: log number of training/validation samples

* feat(wandb): avoid download if file already synced

* feat(wandb): add number of items to metadata

* fix(wandb): allow force sync

* feat(wandb): job -> fine-tune

* refactor(wandb): use show_individual_warnings

* feat(wandb): Logger -> WandbLogger

* feat(wandb): retrive number of items from artifact

* doc(wandb): add link to documentation

Author: borisdayma

README.md

@@ -76,6 +76,7 @@ search = openai.Engine(id="deployment-namme").search(documents=["White House", "
 # print the search
 print(search)
 ```
+
 Please note that for the moment, the Microsoft Azure endpoints can only be used for completion and search operations.
 
 ### Command-line interface
@@ -142,6 +143,12 @@ Examples of fine tuning are shared in the following Jupyter notebooks:
   - [Step 2: Creating a synthetic Q&A dataset](https://github.com/openai/openai-python/blob/main/examples/finetuning/olympics-2-create-qa.ipynb)
   - [Step 3: Train a fine-tuning model specialized for Q&A](https://github.com/openai/openai-python/blob/main/examples/finetuning/olympics-3-train-qa.ipynb)
 
+Sync your fine-tunes to [Weights & Biases](https://wandb.me/openai-docs) to track experiments, models, and datasets in your central dashboard with:
+
+```bash
+openai wandb sync
+```
+
 For more information on fine tuning, read the [fine-tuning guide](https://beta.openai.com/docs/guides/fine-tuning) in the OpenAI documentation.
 
 ## Requirements

openai/_openai_scripts.py

@@ -4,7 +4,7 @@
 import sys
 
 import openai
-from openai.cli import api_register, display_error, tools_register
+from openai.cli import api_register, display_error, tools_register, wandb_register
 
 logger = logging.getLogger()
 formatter = logging.Formatter("[%(asctime)s] %(message)s")
@@ -39,9 +39,11 @@ def help(args):
     subparsers = parser.add_subparsers()
     sub_api = subparsers.add_parser("api", help="Direct API calls")
     sub_tools = subparsers.add_parser("tools", help="Client side tools for convenience")
+    sub_wandb = subparsers.add_parser("wandb", help="Logging with Weights & Biases")
 
     api_register(sub_api)
     tools_register(sub_tools)
+    wandb_register(sub_wandb)
 
     args = parser.parse_args()
     if args.verbosity == 1:

openai/cli.py

@@ -19,6 +19,7 @@
     write_out_file,
     write_out_search_file,
 )
+import openai.wandb_logger
 
 
 class bcolors:
@@ -535,6 +536,19 @@ def prepare_data(cls, args):
         )
 
 
+class WandbLogger:
+    @classmethod
+    def sync(cls, args):
+        resp = openai.wandb_logger.WandbLogger.sync(
+            id=args.id,
+            n_fine_tunes=args.n_fine_tunes,
+            project=args.project,
+            entity=args.entity,
+            force=args.force,
+        )
+        print(resp)
+
+
 def tools_register(parser):
     subparsers = parser.add_subparsers(
         title="Tools", help="Convenience client side tools"
@@ -954,3 +968,40 @@ def help(args):
     sub = subparsers.add_parser("fine_tunes.cancel")
     sub.add_argument("-i", "--id", required=True, help="The id of the fine-tune job")
     sub.set_defaults(func=FineTune.cancel)
+
+
+def wandb_register(parser):
+    subparsers = parser.add_subparsers(
+        title="wandb", help="Logging with Weights & Biases"
+    )
+
+    def help(args):
+        parser.print_help()
+
+    parser.set_defaults(func=help)
+
+    sub = subparsers.add_parser("sync")
+    sub.add_argument("-i", "--id", help="The id of the fine-tune job (optional)")
+    sub.add_argument(
+        "-n",
+        "--n_fine_tunes",
+        type=int,
+        default=None,
+        help="Number of most recent fine-tunes to log when an id is not provided. By default, every fine-tune is synced.",
+    )
+    sub.add_argument(
+        "--project",
+        default="GPT-3",
+        help="""Name of the project where you're sending runs. By default, it is "GPT-3".""",
+    )
+    sub.add_argument(
+        "--entity",
+        help="Username or team name where you're sending runs. By default, your default entity is used, which is usually your username.",
+    )
+    sub.add_argument(
+        "--force",
+        action="store_true",
+        help="Forces logging and overwrite existing wandb run of the same fine-tune.",
+    )
+    sub.set_defaults(force=False)
+    sub.set_defaults(func=WandbLogger.sync)