update readme (#79)

* update readme

* update readme

* add hyper links

Author: ChuckHend

.github/workflows/extension_ci.yml

@@ -11,11 +11,13 @@ on:
       - main
     paths-ignore:
       - 'README.md'
+      - docs/**
   push:
     branches:
       - main
     paths-ignore:
       - 'README.md'
+      - docs/**
 
   release:
     types:

.github/workflows/extension_upgrade.yml

@@ -9,12 +9,15 @@ on:
   pull_request:
     branches:
       - main
+    paths:
+      - ".github/workflows/extension_upgrade.yml"
+      - "./extension"
   push:
     branches:
       - main
-  release:
-    types:
-      - created
+    paths:
+      - ".github/workflows/extension_upgrade.yml"
+      - "./extension"
 
 jobs:
   test:

.gitignore

@@ -10,3 +10,5 @@ pgvector
 test.sql
 META.json
 /vectorize-*
+site/
+poetry.lock

README.md

@@ -21,6 +21,9 @@ This project relies heavily on the work by [pgvector](https://github.com/pgvecto
 [![PGXN version](https://badge.fury.io/pg/vectorize.svg)](https://pgxn.org/dist/vectorize/)
 [![OSSRank](https://shields.io/endpoint?url=https://ossrank.com/shield/3815)](https://ossrank.com/p/3815)
 
+
+pg_vectorize powers the [VectorDB Stack](https://tembo.io/docs/tembo-stacks/vector-db) on [Tembo Cloud](https://cloud.tembo.io/) and is available in all hobby tier instances.
+
 **API Documentation**: https://tembo-io.github.io/pg_vectorize/
 
 **Source**: https://github.com/tembo-io/pg_vectorize
@@ -38,7 +41,7 @@ This project relies heavily on the work by [pgvector](https://github.com/pgvecto
 - [Installation](#installation)
 - [Vector Search Example](#vector-search-example)
 - [RAG Example](#rag-example)
-- [Trigger based updates](#trigger-based-updates)
+- [Updating Embeddings](#updating-embeddings)
 - [Try it on Tembo Cloud](#try-it-on-tembo-cloud)
 
 ## Installation
@@ -128,7 +131,8 @@ SELECT vectorize.table(
     "table" => 'products',
     primary_key => 'product_id',
     columns => ARRAY['product_name', 'description'],
-    transformer => 'sentence-transformers/multi-qa-MiniLM-L6-dot-v1'
+    transformer => 'sentence-transformers/multi-qa-MiniLM-L6-dot-v1',
+    schedule => 'realtime'
 );
 ```
 
@@ -201,9 +205,15 @@ SELECT vectorize.rag(
 "A pencil is an item that is commonly used for writing and is known to be most effective on paper."
 ```
 
-## Trigger based updates
+## Updating Embeddings
+
+When the source text data is updated, how and when the embeddings are updated is determined by the value set to the `schedule` parameter in `vectorize.table` and `vectorize.init_rag`.
+
+The default behavior is `schedule => '* * * * *'`, which means the background worker process checks for changes every minute, and updates the embeddings accordingly. This method requires setting the `updated_at_col` value to point to a colum on the table indicating the time that the input text columns were last changed. `schedule` can be set to any cron-like value.
 
-When vectorize job is set up as `realtime` (the default behavior, via `vectorize.table(..., schedule => 'realtime')`), vectorize will create triggers on your table that will keep your embeddings up to date. When the text inputs are updated or if new rows are inserted, the triggers handle creating a background job that updates the embeddings. Since the transformation is executed in a background job and the transformer model is invoked in a separate container, there is minimal impact on the performance of the update or insert statement.
+Alternatively, `schedule => 'realtime` creates triggers on the source table and updates embeddings anytime new records are inserted to the source table or existing records are updated.
+
+Statements below would will result in new embeddings being generated either immediately (`schedule => 'realtime'`) or within the cron schedule set in the `schedule` parameter.
 
 ```sql
 INSERT INTO products (product_id, product_name, description)
@@ -213,7 +223,3 @@ UPDATE products
 SET description = 'sling made of fabric, rope, or netting, suspended between two or more points, used for swinging, sleeping, or resting'
 WHERE product_name = 'Hammock';
 ```
-
-## Try it on Tembo Cloud
-
-Try it for yourself! Install with a single click on a Vector DB Stack (or any other instance) in [Tembo Cloud](https://cloud.tembo.io/) today.

docs/api/search.md

@@ -17,8 +17,8 @@ vectorize."table"(
     "update_col" TEXT DEFAULT 'last_updated_at',
     "transformer" TEXT DEFAULT 'text-embedding-ada-002',
     "search_alg" vectorize.SimilarityAlg DEFAULT 'pgv_cosine_similarity',
-    "table_method" vectorize.TableMethod DEFAULT 'append',
-    "schedule" TEXT DEFAULT 'realtime'
+    "table_method" vectorize.TableMethod DEFAULT 'join',
+    "schedule" TEXT DEFAULT '* * * * *'
 ) RETURNS TEXT
 ```
 
@@ -33,8 +33,8 @@ vectorize."table"(
 | update_col | text | Column specifying the last time the record was updated. Required for cron-like schedule. Defaults to `last_updated_at` |
 | transformer | text | The name of the transformer to use for the embeddings. Defaults to 'text-embedding-ada-002'. |
 | search_alg | SimilarityAlg | The name of the search algorithm to use. Defaults to 'pgv_cosine_similarity'. |
-| table_method | TableMethod | The method to use for the table. Defaults to 'append', which adds a column to the existing table. |
-| schedule | text | 'realtime' by default for trigger based updates. accepts a cron-like input for a cron based updates. |
+| table_method | TableMethod | `join` to store embeddings in a new table in the vectorize schema. `append` to create columns for embeddings on the source table. Defaults to `join`. |
+| schedule | text | Accepts a cron-like input for a cron based updates. Or `realtime` to set up a trigger. |
 
 ### Sentence-Transformer Examples
 

docs/examples/scheduling.md

@@ -0,0 +1,18 @@
+# Scheduling Embedding Updates
+
+When the source text data is updated, how and when the embeddings are updated is determined by the value set to the `schedule` parameter in `vectorize.table` and `vectorize.init_rag`.
+
+The default behavior is `schedule => '* * * * *'`, which means the background worker process checks for changes every minute, and updates the embeddings accordingly. This method requires setting the `updated_at_col` value to point to a colum on the table indicating the time that the input text columns were last changed. `schedule` can be set to any cron-like value.
+
+Alternatively, `schedule => 'realtime` creates triggers on the source table and updates embeddings anytime new records are inserted to the source table or existing records are updated.
+
+Statements below would will result in new embeddings being generated either immediately (`schedule => 'realtime'`) or within the cron schedule set in the `schedule` parameter.
+
+```sql
+INSERT INTO products (product_id, product_name, description)
+VALUES (12345, 'pizza', 'dish of Italian origin consisting of a flattened disk of bread');
+
+UPDATE products
+SET description = 'sling made of fabric, rope, or netting, suspended between two or more points, used for swinging, sleeping, or resting'
+WHERE product_name = 'Hammock';
+```

docs/examples/sentence_transformers.md

@@ -57,7 +57,8 @@ SELECT vectorize.table(
     "table" => 'products',
     primary_key => 'product_id',
     columns => ARRAY['product_name', 'description'],
-    transformer => 'sentence-transformers/multi-qa-MiniLM-L6-dot-v1'
+    transformer => 'sentence-transformers/multi-qa-MiniLM-L6-dot-v1',
+    scheduler => 'realtime'
 );
 ```
 

mkdocs.yml

@@ -23,6 +23,7 @@ nav:
     - Search:
       - 'examples/sentence_transformers.md'
       - 'examples/openai_embeddings.md'
+      - 'examples/scheduling.md'
 markdown_extensions:
 - toc:
     permalink: true