Skip to content
This repository was archived by the owner on Jan 16, 2026. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
199 changes: 199 additions & 0 deletions docs/docs/pages/sdk/examples/custom-derivation-pipeline.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,199 @@
import { Callout } from 'vocs/components'

# Custom Derivation Pipeline Stage

Extend Kona's derivation pipeline by wrapping the top-level `AttributesQueue` stage with custom logic for monitoring, validation, or transformation.

## Core Concepts

The derivation pipeline uses a stage-based architecture where each stage wraps the previous one:

```
L1Traversal → L1Retrieval → FrameQueue → ChannelProvider →
ChannelReader → BatchStream → BatchProvider → AttributesQueue
```

### Key Traits

Custom stages that wrap the `AttributesQueue` must implement:
- `NextAttributes` - Provides payload attributes for block building
- `OriginProvider` - Provides current L1 origin
- `SignalReceiver` - Handles pipeline resets
- `OriginAdvancer` - Advances L1 origin

## Example: Monitoring Stage

Wrap the `AttributesQueue` to add metrics tracking:

```rust
use kona_derive::{
NextAttributes, OriginProvider, SignalReceiver, OriginAdvancer,
PipelineResult, Signal, OpAttributesWithParent
};
use kona_protocol::{BlockInfo, L2BlockInfo};
use async_trait::async_trait;
use std::time::Instant;

#[derive(Debug)]
pub struct LoggingStage<S> {
inner: S,
attributes_count: u64,
last_origin: Option<BlockInfo>,
}

impl<S> LoggingStage<S> {
pub fn new(inner: S) -> Self {
Self {
inner,
attributes_count: 0,
last_origin: None,
}
}
}

#[async_trait]
impl<S> NextAttributes for LoggingStage<S>
where
S: NextAttributes + Send + Sync,
{
async fn next_attributes(
&mut self,
parent: L2BlockInfo
) -> PipelineResult<OpAttributesWithParent> {
let start = Instant::now();

// Delegate to inner stage
let attributes = self.inner.next_attributes(parent).await?;

// Track metrics
self.attributes_count += 1;
let duration = start.elapsed();

info!(
Copy link

Copilot AI Aug 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code example uses the info! macro without importing or explaining the logging framework. Add an import statement like use log::info; or use tracing::info; to make the example complete and executable.

Copilot uses AI. Check for mistakes.
target: "pipeline::logging",
count = self.attributes_count,
duration_ms = duration.as_millis(),
parent_hash = ?parent.block_info.hash,
"Generated attributes"
);

Ok(attributes)
}
}

impl<S> OriginProvider for LoggingStage<S>
where
S: OriginProvider,
{
fn origin(&self) -> Option<BlockInfo> {
self.inner.origin()
}
}

#[async_trait]
impl<S> SignalReceiver for LoggingStage<S>
where
S: SignalReceiver + Send + Sync,
{
async fn signal(&mut self, signal: Signal) -> PipelineResult<()> {
info!(target: "pipeline::logging", ?signal, "Received signal");
Copy link

Copilot AI Aug 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the previous logging issue, the info! macro usage needs proper imports to be a complete working example.

Copilot uses AI. Check for mistakes.

// Track origin changes on reset
if let Signal::Reset(reset) = &signal {
self.last_origin = Some(reset.l1_origin);
self.attributes_count = 0; // Reset counter
}

self.inner.signal(signal).await
}
}

#[async_trait]
impl<S> OriginAdvancer for LoggingStage<S>
where
S: OriginAdvancer + Send + Sync,
{
async fn advance_origin(&mut self) -> PipelineResult<()> {
let prev_origin = self.inner.origin();
self.inner.advance_origin().await?;
let new_origin = self.inner.origin();

if prev_origin != new_origin {
info!(
target: "pipeline::logging",
prev = ?prev_origin,
new = ?new_origin,
"Advanced origin"
);
}

Ok(())
}
}
```

<Callout type="info">
Custom stages wrap the `AttributesQueue` (top-level stage). For deeper pipeline modifications, you'd need to rebuild the entire pipeline.
</Callout>

## Integration

```rust
use kona_derive::{PipelineBuilder, DerivationPipeline};
use kona_node::{StatefulAttributesBuilder};
use alloc::sync::Arc;

// Build standard pipeline
let pipeline = PipelineBuilder::new()
.rollup_config(rollup_config.clone())
.origin(origin)
.chain_provider(chain_provider)
.l2_chain_provider(l2_chain_provider.clone())
.dap_source(dap_source)
.builder(attributes_builder)
.build_polled();

// Wrap with monitoring
let monitoring_stage = LoggingStage::new(pipeline.attributes);

// Create new pipeline
let custom_pipeline = DerivationPipeline::new(
monitoring_stage,
rollup_config,
l2_chain_provider,
);
```

## Testing

```rust
#[cfg(test)]
mod tests {
use super::*;
use kona_derive::test_utils::TestNextAttributes;

#[tokio::test]
async fn test_logging_stage() {
let mock_inner = TestNextAttributes::new();
let mut stage = LoggingStage::new(mock_inner);

// Test attributes generation
let parent = L2BlockInfo::default();
let result = stage.next_attributes(parent).await;
assert!(result.is_ok());
assert_eq!(stage.attributes_count, 1);

// Test signal handling
let signal = Signal::Reset(Default::default());
stage.signal(signal).await.unwrap();
assert_eq!(stage.attributes_count, 0);
}
}
```

## Related Resources

- [kona-derive](https://github.com/op-rs/kona/tree/main/crates/protocol/derive) - Core derivation pipeline
- [Pipeline Traits](https://github.com/op-rs/kona/tree/main/crates/protocol/derive/src/traits) - Trait definitions
- [Stage Examples](https://github.com/op-rs/kona/tree/main/crates/protocol/derive/src/stages) - Built-in stages
- [OP Stack Derivation Spec](https://specs.optimism.io/protocol/derivation.html) - Protocol specification
4 changes: 3 additions & 1 deletion docs/docs/pages/sdk/examples/intro.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
Examples for working with `kona` crates.

- [Load a Rollup Config for a Chain ID](/sdk/examples/load-a-rollup-config)
- [Create a new L1BlockInfoTx Hardfork Variant](/sdk/examples/new-l1-block-info-tx-hardfork)
- [Transform Frames to a Batch](/sdk/examples/frames-to-batch)
- [Transform a Batch to Frames](/sdk/examples/batch-to-frames)
- [Create a new L1BlockInfoTx Hardfork Variant](/sdk/examples/new-l1-block-info-tx-hardfork)
Copy link

Copilot AI Aug 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The L1BlockInfoTx hardfork variant link appears twice in the list (lines 6 and 8). Remove the duplicate entry on line 6 to maintain a clean, non-redundant documentation structure.

Suggested change
- [Create a new L1BlockInfoTx Hardfork Variant](/sdk/examples/new-l1-block-info-tx-hardfork)

Copilot uses AI. Check for mistakes.
- [Create a new `kona-executor` test fixture](/sdk/examples/executor-test-fixtures)
- [Configuring P2P Network Peer Scoring](/sdk/examples/p2p-peer-scoring)
- [Custom Derivation Pipeline with New Stage](/sdk/examples/custom-derivation-pipeline)
3 changes: 2 additions & 1 deletion docs/sidebar.ts
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,8 @@ export const sidebar: SidebarItem[] = [
{ text: "Transform a Batch into Frames", link: "/sdk/examples/batch-to-frames" },
{ text: "Create a new L1BlockInfoTx Hardfork Variant", link: "/sdk/examples/new-l1-block-info-tx-hardfork" },
{ text: "Create a new kona-executor test fixture", link: "/sdk/examples/executor-test-fixtures" },
{ text: "Configuring P2P Network Peer Scoring", link: "/sdk/examples/p2p-peer-scoring" }
{ text: "Configuring P2P Network Peer Scoring", link: "/sdk/examples/p2p-peer-scoring" },
{ text: "Custom Derivation Pipeline Stage", link: "/sdk/examples/custom-derivation-pipeline" }
]
}
]
Expand Down
Loading