docs: support notebooks (explodinggradients#182)

Author: jjmachan

.readthedocs.yml

@@ -20,7 +20,6 @@ sphinx:
 
 python:
   install:
+    - requirements: ./requirements/docs.txt
     - method: pip
       path: .
-      extra_requirements:
-        - docs

docs/_static/css/ragas.css

@@ -1,25 +1,65 @@
-/* Make pandas tables look correct in dark-mode */
-div.cell_output table {
-    color: var(--color-content-foreground);
+/* Base Theme */
+div.cell_input {
+    background: var(--background) !important;
 }
 
-div.cell_output table {
-    margin: auto;
+.cell_output .output.text_plain,
+.cell_output .output.stream {
+  background: var(--background);
+  border: 1px solid grey;
+  border-radius: var(--mystnb-source-border-radius);
 }
 
-div.cell_output tbody tr:nth-child(odd):not(:hover) {
-    background: var(--color-table-header-background);
+.cell_output .output.stderr {
+  background: var(--background);
+  border: 1px solid #fbc7c7;
+  border-radius: var(--mystnb-source-border-radius);
+}
+
+.cell_output .output.traceback {
+  background: var(--background);
+  border: 1px solid #fbc7c7;
+  border-radius: var(--mystnb-source-border-radius);
+}
+
+/* Pandas tables. Pulled from the Jupyter / nbsphinx CSS */
+div.cell_output table {
+  display: block;
+  overflow-x: auto;
+  white-space: nowrap;
+  border: none;
+  border-collapse: collapse;
+  border-spacing: 0;
+  color: var(--foreground);
+  font-size: 1em;
+  table-layout: fixed;
 }
 
 div.cell_output thead {
-    border-bottom-color: var(--color-code-foreground);
+  border-bottom: 1px solid black;
+  vertical-align: bottom;
 }
 
-div.cell_input {
-    display: none;
+div.cell_output tr,
+div.cell_output th,
+div.cell_output td {
+  text-align: right;
+  vertical-align: middle;
+  padding: 0.5em 0.5em;
+  line-height: normal;
+  white-space: normal;
+  max-width: none;
+  border: none;
+}
+
+div.cell_output th {
+  font-weight: bold;
+}
+
+div.cell_output tbody tr:nth-child(odd) {
+  background: var(--backgound);
 }
 
-.dark {
-    background: var(--color-content-background);
-    color: var(--color-content-foreground);
+div.cell_output tbody tr:hover {
+  background: rgba(66, 165, 245, 0.2);
 }

docs/howtos/assets/langsmith-dataset.png docs/_static/langsmith-dataset.png

@@ -1,5 +1,6 @@
+(user-feedback)=
 # Utilizing User Feedback
 
 User feedback can often be noisy and challenging to harness effectively. However, within the feedback, valuable signals exist that can be leveraged to iteratively enhance your LLM and RAG applications. These signals have the potential to be amplified effectively, aiding in the detection of specific issues within the pipeline and preventing recurring errors. Ragas is equipped to assist you in the analysis of user feedback data, enabling the discovery of patterns and making it a valuable resource for continual improvement.
 
-[Talk to founders](https://calendly.com/shahules/30min) to add this to your LLM app building cycle.
+[Talk to founders](https://calendly.com/shahules/30min) to add this to your LLM app building cycle.

docs/howtos/assets/langsmith-evaluation.png docs/_static/langsmith-evaluation.png

@@ -10,4 +10,56 @@ testset_generation
 feedback
 :::
 
-Ragas is using 
+Ragas aims to create an open standard, providing devs with the tools and techniques to leverage continual learning in their RAG applications. With Ragas, you would be able to
+
+1. Synthetically generate a diverse test dataset that you can use to evaluate your app.
+2. use advanced metrics we built to measure how your app performs.
+3. help monitor your apps in production with custom models and see if there are any discrepancies.
+4. bring up those discrepancies and build new datasets so that you can test and refine your app further to solve them.
+
+(what-is-rag)=
+:::{dropdown} what is RAG and continual learning?
+```{rubric} RAG
+```
+
+Retrieval Augmented Generation (RAG) is a natural language processing (NLP) technique that combines the strengths of retrieval- and generative-based artificial intelligence (AI) models. 
+ RAG uses an information retrieval system to provide data to a Large Language Model (LLM). 
+ RAG models first use a retriever to identify a set of relevant documents from a knowledge base. 
+ RAG can provide more accurate results to queries than a generative LLM on its own because RAG uses knowledge external to data already contained in the LLM.
+
+```{rubric} Continual Learning
+```
+
+With continual learning, models continuously learn and evolve based on the input of increasing amounts of data while retaining previously-learned knowledge. The goal is to develop autonomous agents that can learn continuously and adaptively to develop skills necessary for performing more complex tasks without forgetting what has been learned before. 
+
+The goal of continual learning is to: 
+- Use data that is coming in to automatically retrain the model
+- Gain high accuracy and retain high performing models
+:::
+
+::::{grid} 2
+
+:::{grid-item-card} Metrics Driven Development
+:link: mdd
+:link-type: ref
+What is MDD?
+:::
+
+:::{grid-item-card} Ragas Metrics
+:link: ragas-metrics
+:link-type: ref
+What metrics are available? How do they work?
+:::
+
+:::{grid-item-card} Synthetic Test Data Generation
+:link: testset-generation
+:link-type: ref
+How to create more datasets to test on?
+:::
+
+:::{grid-item-card} Utilizing User Feedback
+:link: user-feedback
+:link-type: ref
+How to leverage the signals from user to improve?
+:::
+::::

docs/howtos/assets/langsmith-feedback.png docs/_static/langsmith-feedback.png

@@ -1,3 +1,4 @@
+(ragas-metrics)=
 # Metrics
 
 ## Component-Wise Evaluation
@@ -32,4 +33,4 @@ semantic_similarity
 answer_correctness
 critique
 
-```
+```

docs/howtos/assets/langsmith-ragas-chain-trace.png docs/_static/langsmith-ragas-chain-trace.png

@@ -1,3 +1,4 @@
+(mdd)=
 # Metrics-Driven Development
 
 While creating a fundamental LLM application may be straightforward, the challenge lies in its ongoing maintenance and continuous enhancement. Ragas' vision is to facilitate the continuous improvement of LLM and RAG applications by embracing the ideology of Metrics-Driven Development (MDD).
@@ -8,4 +9,4 @@ Our mission is to establish an open-source standard for applying MDD to LLM and
 
 - [**Evaluation**](../getstarted/evaluation.md): This enables you to assess LLM applications and conduct experiments in a metric-assisted manner, ensuring high dependability and reproducibility.
 
-- [**Monitoring**](../getstarted/monitoring.md): It allows you to gain valuable and actionable insights from production data points, facilitating the continuous improvement of the quality of your LLM application.
+- [**Monitoring**](../getstarted/monitoring.md): It allows you to gain valuable and actionable insights from production data points, facilitating the continuous improvement of the quality of your LLM application.

docs/concepts/feedback.md

@@ -1,4 +1,5 @@
-# Synthetic Data generation 
+(testset-generation)=
+# Synthetic Test Data generation 
 
 ## Why synthetic test data?
 
@@ -100,4 +101,4 @@ g.set_title("Question type distribution",fontdict = { 'fontsize': 20})
 
 <p align="left">
 <img src="../_static/imgs/question_types.png" alt="test-outputs" width="450" height="400" />
-</p>
+</p>

docs/concepts/index.md

@@ -3,13 +3,16 @@
 
 from sphinxawesome_theme import ThemeOptions
 
+from ragas import __version__
+
 # -- Project information -----------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
 
 project = "ragas"
 copyright = "2023, ExplodingGradients"
 author = "ExplodingGradients"
 release = "0.0.16"
+print("RAGAS VERSION", __version__)
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
@@ -25,15 +28,15 @@
     "sphinx.ext.autodoc",
     "sphinx.ext.napoleon",
     "sphinx.ext.autosummary",
+    "sphinx_design",
     "sphinxawesome_theme.highlighting",
     "sphinxawesome_theme.docsearch",
-    "myst_parser",
-    "sphinx_design",
+    "myst_nb",
 ]
 
 source_suffix = [".rst", ".md"]
 templates_path = ["_templates"]
-exclude_patterns = []
+exclude_patterns = ["_build"]
 myst_number_code_blocks = ["typescript"]
 
 # algolia search
@@ -105,3 +108,4 @@
 
 # -- Myst NB Config -------------------------------------------------
 nb_execution_mode = "off"
+nb_execute_in_temp = True