diff --git a/docs/src/dev-guide/design-kv-ir-streams/background.md b/docs/src/dev-guide/design-kv-ir-streams/background.md
new file mode 100644
index 0000000000..580077cbfd
--- /dev/null
+++ b/docs/src/dev-guide/design-kv-ir-streams/background.md
@@ -0,0 +1,350 @@
+# Background
+
+To understand the KV-IR stream format, we first need to review the following:
+
+* [How clp-s compresses log events](#clp-s-compression), since the process for KV-IR streams is
+ similar but makes a different trade-off between resource usage, compression ratio, and search
+ performance.
+* [How clp-s parses and encodes unstructured text values](#parsing--encoding-unstructured-text).
+
+We discuss each below.
+
+## clp-s compression
+
+At a high-level, [clp-s](../../user-guide/core-clp-s.md) compresses log events into what we call
+archives. Depending on the configured size-threshold for each archive, a set of log events may be
+compressed into one or more archives. The archive format is designed so that each archive is
+self-contained and independent, allowing archives to be searched concurrently.
+
+To compress a log event into an archive, clp-s needs to do the following:
+
+1. [Compute the event's schema](#computing-a-log-events-schema)
+2. [Encode the event's schema](#encoding-log-event-schemas)
+3. [Encode the event's values](#encoding-log-event-values)
+4. [Store the event's encoded values](#storing-encoded-values)
+
+Finally, when all events for an archive have been processed, clp-s needs to serialize and
+[write the archive's data structures](#writing-archives-to-disk). The goal of this process is to
+transform the log events into a form that's more compact to store and faster to search.
+
+:::{tip}
+To learn more about clp-s, check out the original [research paper][clp-s-paper].
+:::
+
+### Computing a log event's schema
+
+A log event's schema is the set of *key* and *value-type* pairs for each KV pair in the log event.
+To compute an event's schema, clp-s iterates over every KV pair in the log event to:
+
+* determine the clp-s value-type that should be assigned to the KV pair.
+* build a tree representation of the schema---what we call a schema tree.
+
+Consider the example log events in [Figure 1](#figure-1), and their schemas in [Table 1](#table-1)
+and [Table 2](#table-2). clp-s' value types, including those used in the schema, are listed in
+[Table 3](#table-3).
+
+clp-s assigns a type to a value based on the value's "abstract" type (i.e., whether it's an integer,
+float, boolean, string, object, null, or array) and how the value should be encoded. For some
+abstract types, clp-s only has one way of encoding it, so it assigns the corresponding clp-s
+type---e.g., the integer corresponding to the `timestamp` key. For other abstract types, clp-s can
+encode the value in multiple ways, so it assigns the clp-s type where the encoded value will result
+in a good trade-off between compactness and efficient searches---e.g., the strings corresponding to
+the `level` and `message` keys use different clp-s types.
+
+Once clp-s assigns a type to a value, it can add it to the log event's schema tree. Except for the
+root, each node in a schema tree represents a unique key and value-type pair from the schema. For
+instance, the tree for the schema in [Table 1](#table-1) is shown in [Figure 2](#figure-2). Since
+the tree represents the structure of a structured log event, each internal (non-leaf) node will
+always correspond to an `Object` or `StructuredArray`, while the leaf nodes will correspond to
+values with primitive types (since `UnstructuredArray` values are encoded as JSON strings, they are
+primitives from the perspective of a schema tree). Accordingly, the root node represents the event
+object itself, and has no key.
+
+(figure-1)=
+:::{card}
+
+```json lines
+{
+ "timestamp": 1744618344394,
+ "level": "info",
+ "message": "task_1 completed successfully. 2 task(s) remain.",
+ "timers": {
+ "stage_1": 0.753,
+ "stage_2": null
+ }
+}
+
+{
+ "timestamp": 1744618344499,
+ "level": "info",
+ "message": "task_2 completed successfully. 1 task(s) remain.",
+ "timers": {
+ "stage_1": 0.945,
+ "stage_2": 0.222
+ }
+}
+```
+
++++
+**Figure 1**: Two JSON log events.
+:::
+
+(table-1)=
+:::{card}
+
+| Key | clp-s value-type |
+|----------------|------------------|
+| timestamp | Integer |
+| level | VarString |
+| message | ClpString |
+| timers | Object |
+| timers.stage_1 | Float |
+| timers.stage_2 | NullValue |
+
++++
+**Table 1**: The schema for log event #1 in [Figure 1](#figure-1). Nested keys are represented
+with dot notation. The value types are described in [Table 3](#table-3).
+:::
+
+(table-2)=
+:::{card}
+
+| Key | clp-s value-type |
+|----------------|------------------|
+| timestamp | Integer |
+| level | VarString |
+| message | ClpString |
+| timers | Object |
+| timers.stage_1 | Float |
+| timers.stage_2 | Float |
+
++++
+**Table 2**: The schema for log event #2 in [Figure 1](#figure-1).
+:::
+
+(table-3)=
+:::{card}
+
+| clp-s value-type | Description | Node type |
+|-------------------|---------------------------------------------------------------------------------------------------------|-----------|
+| Integer | A 64-bit integer | Leaf |
+| Float | A floating-point number | Leaf |
+| Boolean | A boolean | Leaf |
+| VarString | A string without whitespace | Leaf |
+| DateString | A string representing a timestamp | Leaf |
+| ClpString | A string containing whitespace, parsed into an [encoded text AST](#parsing--encoding-unstructured-text) | Leaf |
+| NullValue | A null value | Leaf |
+| UnstructuredArray | An array that's serialized as a JSON string | Leaf |
+| Object | An object | Internal |
+| StructuredArray | An array | Internal |
+
++++
+**Table 3**: clp-s value types.
+:::
+
+(figure-2)=
+::::{card}
+:::{mermaid}
+%%{
+ init: {
+ "theme": "base",
+ "themeVariables": {
+ "primaryColor": "#0066cc",
+ "primaryTextColor": "#fff",
+ "primaryBorderColor": "transparent",
+ "lineColor": "#007fff",
+ "secondaryColor": "#007fff",
+ "tertiaryColor": "#fff"
+ }
+ }
+}%%
+flowchart LR
+ rootObj("<Root>: Object")
+ messageClpStr(""message": ClpString")
+ levelVarStr(""level": VarString")
+ timersObj(""timers": Object")
+ timersStage1Float(""stage_1": Float")
+ timersStage2Null(""stage_2": NullValue")
+ timestampInt(""timestamp": Integer")
+
+ rootObj --> timestampInt
+ rootObj --> levelVarStr
+ rootObj --> timersObj
+ timersObj --> timersStage1Float
+ timersObj --> timersStage2Null
+ rootObj --> messageClpStr
+:::
++++
+**Figure 2**: The schema tree for log event #1 in [Figure 1](#figure-1). Each node's label is of
+the form `"": `, except for the root which doesn't have an explicit name. Each arrow is
+from a parent to a child node.
+::::
+
+### Encoding log event schemas
+
+To compactly encode each event's schema in an archive, clp-s represents each schema with a set of
+integer IDs corresponding to nodes of an archive-level schema tree. This archive-level schema tree
+is built, in part, by merging all events' schema trees and assigning a unique ID to each node. An
+event's schema can then be encoded as the IDs of its *leaf* nodes within the tree, since the leaf
+nodes are sufficient to rebuild the event's tree by traversing from the leaves to the root. For
+instance, [Figure 3](#figure-3) shows the schema tree after adding the example logs
+([Figure 1](#figure-1)) to the tree. The events' schema trees have been merged under the
+`` node. Referencing the leaf node IDs, the schema for event #1 can be
+encoded as `[3, 4, 6, 7, 9]`, corresponding to the schema's leaf nodes.
+
+As [Figure 3](#figure-3) shows, the archive-level schema tree uses different *namespaces* to store
+more than just the KV pairs that appear *in* the event. For instance, the `Metadata` namespace
+contains metadata KV pairs like the log event's index in the archive. The `Default` namespace
+contains the KV pairs that aren't specific to a special namespace, which in the case of Figure 3,
+are the KV pairs that appear in the example log events. As we'll see in future docs, namespaces
+also allow clp-s to compress log events that contain namespaces themselves.
+
+To merge an event's schema tree with the archive-level schema tree, clp-s iterates over each pair of
+nodes---one from each tree:
+
+* If the nodes have the same key and value-type, and all of their predecessor nodes have matching
+ key and value-type pairs, clp-s merges the nodes in the resulting tree.
+* Otherwise, both nodes are added to the resulting tree, and each is assigned a unique integer ID.
+
+
+(figure-3)=
+::::{card}
+:::{mermaid}
+%%{
+ init: {
+ "theme": "base",
+ "themeVariables": {
+ "primaryColor": "#0066cc",
+ "primaryTextColor": "#fff",
+ "primaryBorderColor": "transparent",
+ "lineColor": "#007fff",
+ "secondaryColor": "#007fff",
+ "tertiaryColor": "#fff"
+ }
+ }
+}%%
+flowchart LR
+ root("-1 <Root>")
+ metadataNamespaceRoot("0 <Metadata namespace>: Metadata")
+ logEventIdxInt("1 "log_event_idx": Integer")
+ defaultNamespaceRootObj("2 <Default namespace>: Object")
+ messageClpStr("3 "message": ClpString")
+ levelVarStr("4 "level": VarString")
+ timersObj("5 "timers": Object")
+ timersStage1Float("6 "stage_1": Float")
+ timersStage2Null("7 "stage_2": NullValue")
+ timersStage2Float("8 "stage_2": Float")
+ timestampInt("9 "timestamp": Integer")
+
+ root --> metadataNamespaceRoot
+ metadataNamespaceRoot --> logEventIdxInt
+ root --> defaultNamespaceRootObj
+ defaultNamespaceRootObj --> timestampInt
+ defaultNamespaceRootObj --> levelVarStr
+ defaultNamespaceRootObj --> timersObj
+ timersObj --> timersStage1Float
+ timersObj --> timersStage2Null
+ timersObj --> timersStage2Float
+ defaultNamespaceRootObj --> messageClpStr
+:::
++++
+**Figure 3**: The archive's schema tree after adding the log events from [Figure 1](#figure-1). Each
+node's label is of the form ` : ` except for the namespace nodes which don't have an
+explicit name, and the root which has neither an explicit name nor type.
+::::
+
+
+### Encoding log event values
+
+For each log event, clp-s encodes each value using an encoding method for the value's specific type.
+The goal of each method is to deduplicate any repetitive information (e.g., deduplicating repeated
+`VarString` values with a dictionary) and then represent the value with a 64-bit integer.
+[Table 4](#table-4) lists how clp-s encodes each value type. Most value types are encoded
+conventionally with the following exceptions:
+
+* For the values encoded as dictionary IDs, clp-s simply stores the value in a dictionary and maps
+ it to a unique integer ID.
+* For `ClpString` values, clp-s encodes each component separately.
+* For `NullValue` values, clp-s doesn't need to encode anything since they don't need to be stored
+ explicitly---a `NullValue` leaf node already indicates that the corresponding column of the ERT is
+ null.
+
+(table-4)=
+:::{card}
+
+| clp-s value-type | Encoding |
+|-----------------------------|-------------------------------------------------------------|
+| Integer | 8-byte integer |
+| Float | 8-byte IEEE-754 double-precision float |
+| Boolean | 1-byte integer |
+| VarString | 8-byte dictionary ID |
+| DateString | 8-byte epoch timestamp & 8-byte format string dictionary ID |
+| ClpString | *See below* |
+| --> Format string | 8-byte dictionary ID |
+| --> Encoded variable values | Collection of 8-byte integers |
+| --> String variable values | Collection of 8-byte dictionary IDs |
+| UnstructuredArray | Same as ClpString |
+| NullValue | N/A |
+
++++
+**Table 4**: How clp-s encodes each of its leaf node value types.
+:::
+
+clp-s' two array types are used to encode arrays with different characteristics. `StructuredArray`
+values are similar to `Object` values in that all of their elements will be added to the schema
+tree. Accordingly, this type is more appropriate for encoding arrays whose elements don't change
+types significantly between log events; otherwise, the schema tree would be significantly larger.
+For other arrays, the `UnstructuredArray` type is more appropriate---since it's encoded as a JSON
+string, its elements won't be added to the tree. Nonetheless, values within these arrays can still
+be searched.
+
+### Storing encoded values
+
+clp-s stores a log event's encoded values in a table corresponding to its schema, with one column
+for each node in the schema. We refer to this table as an encoded record table (ERT). By grouping
+events with the same schema into an ERT, clp-s avoids redundantly storing the schema per event
+(unlike, for example, JSON). In addition, ERTs are efficient to search since all columns store
+integers.
+
+### Writing archives to disk
+
+To write an archive's data structures to disk, clp-s serializes them and writes them to one or more
+general-purpose compression streams. Applying general-purpose compression allows us to mitigate some
+of the inefficient encodings (e.g., encoding `Boolean` values as integers) used to maintain
+efficient search performance. For some data structures, like dictionaries, clp-s writes them to disk
+as they are built; yet for other data structures, like the ERTs, clp-s buffers them in memory until
+the archive is complete.
+
+## Parsing & encoding unstructured text
+
+clp-s uses [clp](../../user-guide/core-unstructured/clp.md)'s algorithm to parse and encode
+unstructured text. Unstructured text is a string that contains zero or more variable values
+interspersed with non-variable (static) text. For example, in [Figure 1](#figure-1), log event
+#1's `message` value is unstructured text containing the variable values `task_1` and `2`. At a
+high-level, clp's algorithm uses a set of user-defined regular expressions to match each variable
+value in the unstructured text, decomposing the text into:
+
+* a format string---i.e., the unstructured text with variable values replaced with placeholders.
+* string variable values.
+* encoded variable values---i.e., variable values which have been encoded as 64-bit integers.
+
+Collectively, we refer to these three components as an *encoded text AST*. For instance, log event
+#1's `message` value would be decomposed into the following encoded text AST:
+
+* Format string: `\x12 completed successfully. \x11 task(s) remain.`
+ * `\x12` and `\x11` are variable placeholders representing string and integer variables,
+ respectively.
+* String variable values: `["task_1"]`
+* Encoded variable values: `[1]`
+
+:::{note}
+The clp codebase refers to an encoded text AST's string variable values as "dictionary variables,"
+since they're typically stored in a dictionary. This may change as we update the codebase.
+:::
+
+:::{tip}
+To learn more about clp, check out the original [research paper][clp-paper].
+:::
+
+[clp-paper]: https://www.usenix.org/system/files/osdi21-rodrigues.pdf
+[clp-s-paper]: https://www.usenix.org/system/files/osdi24-wang-rui.pdf
diff --git a/docs/src/dev-guide/design-kv-ir-streams/index.md b/docs/src/dev-guide/design-kv-ir-streams/index.md
new file mode 100644
index 0000000000..ac65299063
--- /dev/null
+++ b/docs/src/dev-guide/design-kv-ir-streams/index.md
@@ -0,0 +1,29 @@
+# KV-IR streams
+
+The key-value pair internal representation (abbreviated as KV-IR) stream format is a storage format
+for dynamically structured (e.g., JSON) logs. Compared to a JSON file, the KV-IR stream format is
+typically smaller and faster for clp-s to compress. Compared to clp-s' archive format, the KV-IR
+stream format buffers less data in-memory, making it more suitable for use in resource-constrained
+environments and low-latency use cases (e.g., logging libraries). This section describes the format
+as well as the key design decisions behind it.
+
+::::{grid} 1 1 1 1
+:gutter: 2
+
+:::{grid-item-card}
+:link: background
+Background
+^^^
+Any necessary information to understand the stream's format.
+:::
+::::
+
+:::{warning}
+🚧 This section is still under construction.
+:::
+
+:::{toctree}
+:hidden:
+
+background
+:::
diff --git a/docs/src/dev-guide/index.md b/docs/src/dev-guide/index.md
index 1dbf560a02..84d3e536bc 100644
--- a/docs/src/dev-guide/index.md
+++ b/docs/src/dev-guide/index.md
@@ -80,5 +80,6 @@ tooling-gh-workflows
:hidden:
design-project-structure
+design-kv-ir-streams/index
design-parsing-wildcard-queries
:::
diff --git a/docs/src/dev-guide/tooling-gh-workflows.md b/docs/src/dev-guide/tooling-gh-workflows.md
index 0f2551bc14..bc38f75257 100644
--- a/docs/src/dev-guide/tooling-gh-workflows.md
+++ b/docs/src/dev-guide/tooling-gh-workflows.md
@@ -21,8 +21,8 @@ shown below.
"primaryColor": "#0066cc",
"primaryTextColor": "#fff",
"primaryBorderColor": "transparent",
- "lineColor": "#9580ff",
- "secondaryColor": "#9580ff",
+ "lineColor": "#007fff",
+ "secondaryColor": "#007fff",
"tertiaryColor": "#fff"
}
}