docs: update for 1.1

AGENTS.md

@@ -14,7 +14,7 @@ Top-level modules:
 - `example_plugins/` — reference plugins (UDFs, output sinks, labels service) using the pluggy-based plugin system. Do not add production code here.
 - `example_rules/` — sample SML rules and YAML config.
 
-Reference files: `docs/DEVELOPMENT.md` (setup), `example_plugins/src/register_plugins.py` (plugin patterns), `example_plugins/src/services/labels_service.py` (labels service example).
+Reference files: `docs/development/local.md` (setup), `example_plugins/src/register_plugins.py` (plugin patterns), `example_plugins/src/services/labels_service.py` (labels service example).
 
 ## Design
 
@@ -157,7 +157,7 @@ uv tool run fawltydeps --check-unused --pyenv .venv
 - Do not disable lint or type rules to silence errors. Fix the underlying issue, or use a narrowly-scoped `# noqa: <code>` / `# type: ignore[<code>]` with a comment explaining why.
 - Before adding a new dependency, check it for known CVEs and confirm the license is compatible with `LICENSE.md`.
 - Do not commit generated protobuf files from an untrusted toolchain; always regenerate via `./gen-protos.sh`.
-- Default Docker bindings are `127.0.0.1`; do not change bind addresses without explicit instruction (see `docs/DEVELOPMENT.md` §6).
+- Default Docker bindings are `127.0.0.1`; do not change bind addresses without explicit instruction (see `docs/development/local.md` §6).
 
 ## Code review
 

CHANGELOG.md

@@ -20,7 +20,7 @@ For more information about each release including git tags and artifacts, see [R
 - `GetActionId()` stdlib UDF for retrieving the current action ID from execution context ([#327](https://github.com/roostorg/osprey/pull/327) by [@julietshen](https://github.com/julietshen))
 - `ParseInt` UDF for converting numeric strings to integers ([#190](https://github.com/roostorg/osprey/pull/190) by [@bealsbe](https://github.com/bealsbe))
 - `StringSlice` UDF for extracting substrings by index range ([#189](https://github.com/roostorg/osprey/pull/189) by [@bealsbe](https://github.com/bealsbe))
-- `InExperiment` UDF for checking experiment membership ([#203](https://github.com/roostorg/osprey/pull/203) by [@bealsbe](https://github.com/bealsbe))
+- `Experiment` and `ExperimentWhen` UDFs for checking experiment membership ([#203](https://github.com/roostorg/osprey/pull/203) by [@bealsbe](https://github.com/bealsbe))
 - Per-action health metrics in the executor ([#191](https://github.com/roostorg/osprey/pull/191) by [@cmttt](https://github.com/cmttt))
 - Option to suppress cached errors to reduce metric bloat ([#180](https://github.com/roostorg/osprey/pull/180) by [@lithium-powered](https://github.com/lithium-powered))
 - Experimental asyncio-native worker with metrics and engine/coordinator improvements ([#341](https://github.com/roostorg/osprey/pull/341) by [@cmttt](https://github.com/cmttt))

README.md

@@ -2,20 +2,31 @@
 
 # Osprey
 
-**Automate the obvious and investigate the ambiguous.** High-performance safety rules engine for real-time event processing at scale. 
+**Automate the obvious and investigate the ambiguous.** Osprey is a safety rules engine and investigation console for real-time event processing at scale.
 
-- take action based on user behavior
-- combine actions with human written rules
-- let operators query human actions and past decisions
-- perform investigations or write new rules based on decisions
+Your platform streams events to Osprey, and human-written rules evaluate each one as it arrives—taking automatic action, applying labels to the entities involved, and sending verdicts and custom effects to your own systems. Analysts then query and chart the results to spot patterns, investigate, and turn what they find into new rules. Originally developed internally at [Discord](https://discord.com/) to combat spam, abuse, botting, and scripting across its platform, Osprey has been open-sourced to help other platforms facing similar challenges.
 
-Osprey is an event stream decisions engine and analysis UI designed to investigate and take automatic action on events and their properties as they happen in real-time. Originally developed internally at [Discord](https://discord.com/) to combat spam, abuse, botting, and scripting across its platform, Osprey has been open-sourced to help other platforms facing similar challenges.
+![The Query page with an SML filter for post-creation events and its history of past queries, beside a timeseries chart showing matching event volume in fifteen-minute buckets](docs/images/query-and-charts.png)
 
-![Osprey UI sample](docs/images/query-and-charts.png)
+Rules are written in SML, Osprey's structured rule language, and extended with user-defined functions (UDFs). Osprey tracks state across events by labeling entities when you provide a labels service backend; see [labels_service.py](./example_plugins/src/services/labels_service.py) for a Postgres-backed example.
 
-Osprey is a library for processing actions through human-written rules and outputting verdicts & custom effects back to configurable output sinks. It evaluates events using structured rule logic (SML) that is extendable via user-defined functions (UDFs). Osprey can also track state across events by labelling entities if implementers provide a labels service backend (see [labels_service.py](./example_plugins/src/services/labels_service.py) for a Postgres-backed labels service example).
+Osprey is built for engineers and Trust & Safety teams who want to explore, test, and integrate it into their platform for incident response and investigations. [Read more about user research and personas](docs/research-personas.md).
 
-Osprey is built for engineers and Trust & Safety teams who want to explore, test, and integrate its core capabilities into their platform for incident response and Trust & Safety investigation. [Read more about user research and personas](docs/user_personas.md).
+## Try it
+
+If you have Docker with Compose v2, one command brings up the full stack with sample data and opens the UI on a pre-filled query:
+
+```sh
+./demo.sh
+```
+
+Or, without cloning the repo first:
+
+```sh
+curl -sSL https://raw.githubusercontent.com/roostorg/osprey/main/demo.sh | bash
+```
+
+The [Getting Started guide](https://roostorg.github.io/osprey/latest/development/) explains what the demo starts, what to try in the UI, and how to shut it all down. The rest of the documentation lives at [roostorg.github.io/osprey/latest](https://roostorg.github.io/osprey/latest/).
 
 ## Adopters
 
@@ -28,21 +39,19 @@ Using Osprey and want to add your project/organization to this list? [Open a pul
 
 ## Development
 
-- See the [development guide](./docs/development/) for comprehensive development setup and workflow documentation
-- All code changes should pass linting (Ruff) and type checking (MyPy)
-- Pre-commit hooks automatically run on each commit to maintain code quality
+See the [development guide](./docs/development/) for development setup and workflow documentation, including the linting, type checking, and pre-commit hooks that changes are expected to pass.
 
-## Join Us
+## Join us
 
-Writing code is not the only way to help the project. Reviewing pull requests, answering questions to help others on mailing lists or issues, providing feedback from a domain expert perspective, organizing and teaching tutorials, working on the website, improving the documentation, are all priceless contributions.
+Writing code is not the only way to help the project. Reviewing pull requests, answering questions in issues and discussions, providing feedback from a domain expert perspective, teaching tutorials, and improving the documentation are all priceless contributions.
 
 - Join us in [our Discord server](https://discord.gg/5Csqnw2FSQ)
 - Join our [newsletter](https://roost.tools/#get-started) for more announcements and information
 - Follow us on [Bluesky](https://bsky.app/profile/roost.tools) or [LinkedIn](https://www.linkedin.com/company/roost-tools/)
 
 _[ROOST](https://roost.tools) (Robust Open Online Safety Tools) is a non-profit organization that brings together expertise, resources, and investments from major technology companies and philanthropies to build scalable, interoperable safety infrastructure for the AI era._
 
-### Feedback Wanted
+### Feedback wanted
 
 This is a working system, not a prototype. Try it locally, connect your data, write some rules, and tell us what's missing for your use case. We're particularly interested in:
 
@@ -51,8 +60,8 @@ This is a working system, not a prototype. Try it locally, connect your data, wr
 - Missing detection capabilities or response actions you need
 - API improvements that would make adoption easier for your team
 
-Your experimentation feedback will directly shape future priorities and help us build the most useful Trust & Safety tooling for the community.
+Your feedback will directly shape future priorities and help us build the most useful Trust & Safety tooling for the community.
 
 ## Recognition
 
-Discord uses Osprey to quickly detect and remove new types of harm that put users at risk. Rather than leaving other platforms to build similar tools from scratch, ROOST and Discord have open-sourced this powerful rule engine in collaboration with [internet.dev](https://internet.dev/) to make it available for anyone who needs it.
+Discord uses Osprey to quickly detect and remove new types of harm that put users at risk. Rather than leaving other platforms to build similar tools from scratch, ROOST and Discord have open-sourced Osprey in collaboration with [internet.dev](https://internet.dev/) to make it available for anyone who needs it.

demo.sh

@@ -298,17 +298,14 @@ echo -e "  • Osprey UI:      ${GREEN}${UI_URL}${NC}"
 echo -e "  • Druid Console:  ${GREEN}http://localhost:8888${NC}"
 echo -e "  • Osprey API:     ${GREEN}http://localhost:5004${NC}"
 echo ""
-echo -e "${BLUE}Demo Rules Active:${NC}"
-echo -e "  • ContainsHello  - Bans users who say 'hello'"
-echo -e "  • LazyPostRule   - Labels posts with 'lazy' as low_effort"
-echo -e "  • QuickPostRule  - Labels posts with 'quick' as potential_bot"
-echo -e "  • FoxPostRule    - Bans users who say 'fox' (spam pattern)"
+echo -e "${BLUE}Demo Rule Active:${NC}"
+echo -e "  • ContainsHello  - Bans users who post the word 'hello' and labels them 'meow'"
 echo ""
 echo -e "${BLUE}What to Demo:${NC}"
 echo -e "  1. Event Stream  - See processed events with rule matches"
 echo -e "  2. TopN Panel    - See users grouped by labels/bans"
 echo -e "  3. Timeseries    - See event volume over time"
-echo -e "  4. Query Filter  - Try: LazyPostRule == True"
+echo -e "  4. Query Filter  - Try: ContainsHello == True"
 echo -e "  5. Rules Viz     - View the rule dependency graph"
 echo ""
 echo -e "${YELLOW}Opening Osprey UI in browser...${NC}"

docs/README.md

@@ -1,14 +1,22 @@
 # Welcome
 
-Welcome to the Osprey documentation! Visit [Osprey on GitHub](https://github.com/roostorg/osprey#readme) for source code and project information. These docs are split into a few guides, depending on who you are and what you're looking for:
+Osprey is an open source rules engine and investigation console for trust and safety teams: your platform streams events to it, your rules evaluate each one in real time, and your analysts query, chart, and act on the results. Originally built at Discord and running in production at Bluesky, Osprey is now developed in the open by [ROOST](https://roost.tools) and the community; visit [Osprey on GitHub](https://github.com/roostorg/osprey#readme) for source code and project information.
 
-- [User Guide](user/): learn about Osprey, its functionality, and the user interface
-- [Development Guide](development/): get Osprey running and learn how to navigate the code
+These docs are split into a few guides, depending on who you are and what you're looking for:
 
-See additional topics for reference:
+- [User Guide](user/): learn the investigation UI—querying events, labeling entities, and running bulk jobs.
 
-- [Writing Rules](rules.md)
-- [User Research & Personas](research-personas.md)
+- [Development Guide](development/): run a local development environment and change Osprey itself.
+
+- [Integration Guide](integration/): connect Osprey to your platform—get data in and out, and extend it with plugins.
+
+- [Concepts](concepts.md): learn about the basic concepts and terminology used in Osprey and these docs.
+
+The docs are versioned; other versions are available at the [documentation site index](https://roostorg.github.io/osprey/).
+
+[![The Query page with an SML filter for post-creation events and its history of past queries, beside a timeseries chart showing matching event volume in fifteen-minute buckets](images/query-and-charts.png)](images/query-and-charts.png)
+
+The fastest way to get a feel for Osprey is the one-command demo in [Getting Started](development/), which brings up the full stack on live sample data in a few minutes.
 
 ## Contributing
 

docs/SUMMARY.md

@@ -10,23 +10,32 @@
     - [Query Syntax](user/investigate/query-syntax.md)
   - [Manage](user/manage.md)
   - [Operate](user/operate.md)
-  - [Appearance](user/appearance.md)
 
 ---
 
 - [Development Guide](development/README.md)
-  - [Workflow](development/workflow.md)
-  - [Tools Overview](development/tools.md)
+  - [Local Development](development/local.md)
+  - [Development Workflow](development/workflow.md)
+  - [Development Tools](development/tools.md)
+  - [CLI Reference](development/cli-reference.md)
   - [Troubleshooting](development/troubleshooting.md)
   - [IDE Setup](development/ide.md)
   - [Release Process](development/releases.md)
 
 ---
 
+- [Integration Guide](integration/README.md)
+  - [Data Flow](integration/data-flow.md)
+  - [Integrations & Plugins](integration/integrations.md)
+
+---
+
 # Concepts & More
 
-- [Writing Rules](rules.md)
-- [Integrations & Plugins](integrations.md)
+- [Concepts](concepts.md)
+- [Writing Rules](rules/README.md)
+  - [Examples](rules/examples.md)
+  - [Why a Bespoke Language?](rules/why-sml.md)
 - [User Research & Personas](research-personas.md)
 
 ---

docs/book.toml

@@ -6,11 +6,19 @@ src = "."
 
 [build]
 build-dir = "./book"
+create-missing = false
 
 [output.html]
-site-url = "/osprey/"
+default-theme = "roost"
+preferred-dark-theme = "roost"
+additional-css = ["theme/css/roost.css", "theme/css/roost-hljs.css", "theme/css/content.css"]
 edit-url-template = "https://github.com/roostorg/osprey/edit/main/docs/{path}"
 git-repository-url = "https://github.com/roostorg/osprey"
+site-url = "/osprey/"
 
 [output.html.print]
 enable = false
+
+[output.html.redirect]
+"/integrations.html" = "integration/integrations.html"
+"/rules.html" = "rules/index.html"

docs/concepts.md

@@ -0,0 +1,58 @@
+# Concepts
+
+Osprey watches a stream of events happening on your platform, runs each one through your rules in real time, and records what it found so analysts can query and act on it. This page defines the terms the rest of these docs (and the UI itself) use. They're listed in the order data flows through Osprey, and later concepts build on earlier ones, so it's worth a read from start to finish.
+
+Throughout the docs are examples of moderating a small social network where users create posts; the same concepts adapt to any platform where people interact.
+
+## Events (and actions)
+
+Anything that happens on your platform—e.g. someone registers an account, creates a post, sends a message, or reacts to content—can be sent to Osprey as an **event**, as it happens. Each event arrives with a name, like `create_post`, and whatever JSON data your platform sent along, e.g. the author's user ID and the post text "hello world."
+
+> [!NOTE]
+> You may also come across the term **action**: Osprey's submission API and rules engine historically call each incoming event an action, which is why the query bar filters on `ActionName` even though the UI's live feed is the Event Stream. In this context, they're the same: one event in, one `ActionName` recorded.
+
+## Features
+
+A **feature** is a named value that your rules extract from each event: `PostText`, `UserId`, `AccountAgeSeconds`. Features are what you query on; every feature your rules define is queryable by name, unless its name starts with an underscore, which keeps it private to the rule file that defined it.
+
+For a "hello world" post, your rules might extract `UserId`, `PostText`, and `EventType` as features.
+
+## Entities
+
+An **entity** is a feature that uniquely identifies something on your platform: a user ID, an email address, an IP address. Entities are declared with a type (like `User`) so Osprey knows which values refer to the same thing across many events. That identity is what makes [labels](#labels) possible, and it's how selecting an entity in the UI can open a view of everything Osprey knows about it.
+
+In our example, `UserId` would be declared as an entity: the same user posts many times, and you want Osprey to remember them.
+
+## Rules
+
+A **rule** is a named condition over features, written in Osprey's rule language, SML. Rules are evaluated against every event as it arrives. On their own, rules just return a boolean true/false (which is itself queryable, like any feature). Rules can be wired to [effects](#effects).
+
+In our example, a rule might cover "the event is a post creation and the post text contains 'hello'."
+
+Rules live as code in your Osprey deployment alongside the Osprey code itself. See [Writing Rules](rules/) for more detail.
+
+## Effects
+
+An **effect** is what a matched rule does beyond evaluating; e.g. ban the user, add a label to an entity, declare a verdict. Effects are recorded on the event's result, so an investigator can always see afterward exactly what Osprey did and why.
+
+When the demo's `ContainsHello` rule matches our "hello world" post, it fires two effects: a ban for the author, and a `meow` label added to them.
+
+## Labels
+
+A **label** is a tag on an entity that persists across events—Osprey's memory. Rules add and remove labels as an effect, and can also _check_ labels, so past decisions inform future ones: "flag this post if its author was previously labeled a spammer." You can also add or remove labels by hand from the UI, one entity at a time or in bulk.
+
+Labels have a name, the entity types they apply to, and a connotation (positive, negative, or neutral). See [Labels](user/investigate/labels.md) for more information about how they're used for investigations.
+
+## Verdicts
+
+A **verdict** is Osprey's answer when a caller is waiting for one. Most deployments feed Osprey from a queue and read results the same way, but a service can also submit an event synchronously and get a response; rules declare verdict strings, and those—along with any labels applied—are what the caller gets back. If your deployment only consumes results asynchronously, you may never handle a verdict directly, though you can spot them among an event's recorded features in the UI.
+
+## Results, and where they surface
+
+Every processed event produces an execution result: the extracted features, the rules that matched, and the effects that fired. Results are indexed for querying—that's what the whole [Investigate](user/investigate/) side of the UI reads. The query bar filters results by feature (`ContainsHello == True`), charts aggregate them over time, Top N groups them by any feature, and the Event Stream shows them one by one as they happen. Clicking any entity opens its details: current labels and its history on your platform.
+
+How results get from the engine to those views (and into your own systems) is plumbing your developers control; the [Data Flow](integration/data-flow.md) page covers it.
+
+## Where to go from here
+
+Try these ideas out on live sample data by running the [demo](development/), then head to [Investigate](user/investigate/) to learn the query workflow, or [Writing Rules](rules/) when you're ready to automate a decision of your own.