Latent Space Tools

Observability & Security for LLMs x Latent Space Apps

The following Software Suite, Visual Artifacts, and any/all concepts herein (hereafter Latent Space Tools) are made available under the Apache 2 license.

Latent Space Tools help conceptualize, visualize, and subsequently operationalize the necessary architecture and software components for secure LLM Deployment & Monitoring.

Architectureal Overview:

Click here for Detailed Annotations

Key Components:

Input Pre-Processing

1) Prompt Injection Detection & Mitigation

2) Service Denial & Performance Monitoring

Data Enrichment, Event-Driven Monitoring & Alerting

3) Topic/Sentiment Modeling x Centroid Vector Comparisions

Output Post-Processing

4) Attack Mitigation, Appending (Un)Certainty & Response Non-Conformity

Output Forecasting

5) Heatmaps x Dimensionality Drift via Conformal Prediction Intervals

Core Concepts:

N-Dimensional Drift:

Given a latent space generally represents a reduced dimensionality from the feature space, we expect the 'aggregate' dimensions to be noisier than their components.

That said, the chosen dimensions should represent meaningful metrics worth monitoring; hence the value in conceptualizing, monitoring, and forecasting changes to those values

Conformal Prediction

Latent Space Tools extensively leverage the concept of conformal prediction; whereby previous outputs better predict future outputs than do Bayesian priors or assumptions