docs/examples/minimal.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Minimal Usage Example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This provides an minimal usage example for `pydantic-cereal`.\n",
    "To start, use the following imports and create a global `cereal` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"Minimal imports to get started.\"\"\"\n",
    "\n",
    "from fsspec import AbstractFileSystem\n",
    "from pydantic import BaseModel, ConfigDict\n",
    "from pydantic_cereal import Cereal\n",
    "\n",
    "cereal = Cereal()  # global variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have a custom type `MyType`, in this case it's just an wrapper for a `str`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MyType(object):\n",
    "    \"\"\"My custom type, which isn't a Pydantic model.\"\"\"\n",
    "\n",
    "    def __init__(self, value: str):\n",
    "        \"\"\"Initialize the object.\"\"\"\n",
    "        self.value = str(value)\n",
    "\n",
    "    def __repr__(self) -> str:\n",
    "        \"\"\"Represent the string.\"\"\"\n",
    "        return f\"MyType({self.value})\"\n",
    "\n",
    "    def __str__(self) -> str:\n",
    "        \"\"\"Return the internal string.\"\"\"\n",
    "        return str(self.value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We must add reader and writer classes for it. \n",
    "These must accept [`fsspec`](https://filesystem-spec.readthedocs.io/en/latest/) file system\n",
    "and path (within that filesystem) as inputs.\n",
    "We can register these with our `cereal` object by creating a wrapped (`Annotated`) type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_reader(fs: AbstractFileSystem, path: str) -> MyType:\n",
    "    \"\"\"Read a MyType from an fsspec URI.\"\"\"\n",
    "    res = fs.read_text(path)\n",
    "    if isinstance(res, bytes):\n",
    "        res = res.decode(\"utf8\")\n",
    "    else:\n",
    "        res = str(res)\n",
    "    return MyType(value=res)\n",
    "\n",
    "\n",
    "def my_writer(obj: MyType, fs: AbstractFileSystem, path: str) -> None:\n",
    "    \"\"\"Write a MyType object to an fsspec URI.\"\"\"\n",
    "    fs.write_text(path, obj.value)\n",
    "\n",
    "\n",
    "MyWrappedType = cereal.wrap_type(MyType, reader=my_reader, writer=my_writer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that your type checker should recognize `MyWrappedType` as exactly `MyType`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import get_args  # noqa\n",
    "\n",
    "assert get_args(MyWrappedType)[0] == MyType"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use this type in a Pydantic model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ExampleModel(BaseModel):\n",
    "    \"\"\"Example model.\"\"\"\n",
    "\n",
    "    model_config = ConfigDict(arbitrary_types_allowed=True)\n",
    "\n",
    "    fld: MyWrappedType  # NOTE: Make sure to use the wrapped type!\n",
    "    value: str = \"default_value\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can instantiate objects as usual:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "mdl = ExampleModel(fld=MyType(\"my_field\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, you can write your model to an arbitrary directory-like `fsspec` URI.\n",
    "In this example, we're writing to a temporary [`MemoryFileSystem`](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.implementations.memory.MemoryFileSystem):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/example_model'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cereal.write_model(mdl, \"memory://example_model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we can load another object from there:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MyType(my_field)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "obj = cereal.read_model(\"memory://example_model\")\n",
    "assert isinstance(obj, ExampleModel)\n",
    "obj.fld"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you require a specific type (or base type), you can specify this in `read_model`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MyType(my_field)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cereal.read_model(\"memory://example_model\", supercls=ExampleModel).fld"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Inspecting the path, you can see the file structure:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fsspec.implementations.memory import MemoryFileSystem  # noqa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "fs = MemoryFileSystem()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['/example_model/9048f517f71f434aad4a5481f3b2b3d4',\n",
       " '/example_model/model.json',\n",
       " '/example_model/model.schema.json']"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fs.glob(\"example_model/*\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "pyd-cereal",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}