show evals summary at end of pytest run

.github/workflows/llm-evaluation.yaml

@@ -43,7 +43,6 @@ jobs:
           OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
           BRAINTRUST_API_KEY: ${{ secrets.BRAINTRUST_API_KEY }}
           UPLOAD_DATASET: "true"
-          PUSH_EVALS_TO_BRAINTRUST: "true"
           EXPERIMENT_ID: github-${{ github.run_id }}.${{ github.run_number }}.${{ github.run_attempt }}
         run: |
           poetry run pytest --no-cov tests/llm/test_ask_holmes.py tests/llm/test_investigate.py -n 6

CLAUDE.md

@@ -137,7 +137,8 @@ poetry run pytest tests/llm/test_ask_holmes.py
 - `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`: LLM API keys
 - `MODEL`: Override default model
 - `RUN_LIVE`: Use live tools in tests instead of mocks
-- `BRAINTRUST_API_KEY`: For test result tracking
+- `BRAINTRUST_API_KEY`: For test result tracking and CI/CD report generation
+- `BRAINTRUST_ORG`: Braintrust organization name (default: "robustadev")
 
 ## Development Guidelines
 

docs/development/evals/index.md

@@ -100,6 +100,8 @@ Configure evaluations using these environment variables:
 | `RUN_LIVE` | `RUN_LIVE=true` | Execute `before-test` and `after-test` commands, ignore mock files |
 | `UPLOAD_DATASET` | `UPLOAD_DATASET=true` | Sync dataset to external evaluation platform |
 | `EXPERIMENT_ID` | `EXPERIMENT_ID=my_baseline` | Custom experiment name for result tracking |
+| `BRAINTRUST_API_KEY` | `BRAINTRUST_API_KEY=sk-...` | Enable Braintrust integration for result tracking and CI/CD report generation |
+| `BRAINTRUST_ORG` | `BRAINTRUST_ORG=my-org` | Braintrust organization name (defaults to "robustadev") |
 
 ### Simple Example
 

docs/development/evals/reporting.md

@@ -40,7 +40,6 @@ export BRAINTRUST_API_KEY=sk-your-api-key-here
 ```bash
 export BRAINTRUST_API_KEY=sk-your-key
 export UPLOAD_DATASET=true
-export PUSH_EVALS_TO_BRAINTRUST=true
 
 pytest ./tests/llm/test_ask_holmes.py
 ```
@@ -58,7 +57,6 @@ pytest -n 10 ./tests/llm/test_*.py
 | Variable | Purpose |
 |----------|---------|
 | `UPLOAD_DATASET` | Sync test cases to Braintrust |
-| `PUSH_EVALS_TO_BRAINTRUST` | Upload evaluation results |
 | `EXPERIMENT_ID` | Name your experiment run. This makes it easier to find and track in Braintrust's UI |
 | `MODEL` | The LLM model for Holmes to use |
 | `CLASSIFIER_MODEL` | The LLM model to use for scoring the answer (LLM as judge) |