This repository was archived by the owner on Apr 6, 2026. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 336
Create chain-monitoring.mdx #764
Merged
Merged
Changes from 9 commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
3d66e2f
Create chain-monitoring.mdx
cpengilly 1d7be24
updates
cpengilly bf6e081
Update chain-monitoring.mdx
cpengilly 21a29f8
Finish up missing content
bradleycamacho 7847749
Add feedback
bradleycamacho 4f393f0
Merge branch 'main' into monitoring-explainer
bradleycamacho 70cc71d
Fix linting
bradleycamacho 8d7ef27
Merge branch 'monitoring-explainer' of https://github.com/ethereum-op…
bradleycamacho 4d3f3f3
Add missing words to words.txt
bradleycamacho a5e801f
Update pages/builders/chain-operators/tools/chain-monitoring.mdx
bradleycamacho 56292ee
Update pages/builders/chain-operators/tools/chain-monitoring.mdx
bradleycamacho b4ca9c2
Update pages/builders/chain-operators/tools/chain-monitoring.mdx
bradleycamacho 6285e5a
Update pages/builders/chain-operators/tools/chain-monitoring.mdx
bradleycamacho ccc7da9
Update pages/builders/chain-operators/tools/chain-monitoring.mdx
bradleycamacho b65d1b8
Update pages/builders/chain-operators/tools/chain-monitoring.mdx
bradleycamacho d8871d7
Update pages/builders/chain-operators/tools/chain-monitoring.mdx
bradleycamacho fbcedfd
Apply review suggestions
bradleycamacho 9901f39
Merge branch 'monitoring-explainer' of https://github.com/ethereum-op…
bradleycamacho 0099edd
Apply linting
bradleycamacho File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
128 changes: 128 additions & 0 deletions
128
pages/builders/chain-operators/tools/chain-monitoring.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| --- | ||
| title: Chain Monitoring Options | ||
| lang: en-US | ||
| description: Learn the basics of onchain and offchain monitoring options for your OP Stack chain. | ||
| --- | ||
|
|
||
| import { Callout } from 'nextra/components' | ||
|
|
||
| # Chain Monitoring Options | ||
|
|
||
| This explainer covers the basics of **onchain** and **offchain** monitoring options for your OP Stack chain. Onchain monitoring services allow chain operators to monitor the overall system and onchain events. | ||
| Offchain monitoring allows chain operators to monitor the operation and behavior of nodes and other offchain components. | ||
|
bradleycamacho marked this conversation as resolved.
Outdated
|
||
|
|
||
| ## Onchain Monitoring Services | ||
|
|
||
| Onchain monitoring services provide insights into the overall system and helps chain operators track and monitor onchain events. Some examples of onchain monitoring services include `monitorism` and `dispute-mon`. | ||
|
|
||
|
bradleycamacho marked this conversation as resolved.
Outdated
|
||
| ### `monitorism` | ||
|
|
||
| Monitorism is a tooling suite that supports monitoring and active remediation actions for the OP Stack chain. Monitorism uses monitors as passive security providing automated monitoring for the OP Stack. They are used to monitor the OP stack and alert on specific events that could be a sign of a security incident. | ||
|
|
||
| Currently. the list of monitors includes: | ||
|
bradleycamacho marked this conversation as resolved.
|
||
|
|
||
| Security integrity monitors: These are monitors necessary for making sure Bridges between L2 and L1 are safe and work as expected. These monitors are divided in two subgroups: | ||
|
|
||
| * Pre-Faultproof Chain Monitors: | ||
| * Fault Monitor: checks for changes in output roots posted to the `L2OutputOracle` contract. On change, reconstructing the output root from a trusted L2 source and looking for a match. | ||
|
bradleycamacho marked this conversation as resolved.
Outdated
|
||
| * Withdrawals Monitor: checks for new withdrawals that have been proven to the OptimismPortal contract. Each withdrawal is checked against the `L2ToL1MessagePasser` contract. | ||
| * Faultproof chain monitors: | ||
| * Faultproof Withdrawal: The Faultproof Withdrawal component monitors `ProvenWithdrawals` events on the `OptimismPortal` contract and performs checks to detect any violations of invariant conditions on the chain. If a violation is detected, it logs the issue and sets a Prometheus metric for the event. This component is designed to work exclusively with chains that are already utilizing the Fault Proofs system. This is a new version of the deprecated chain-mon faultproof-wd-mon. For detailed information on how the component works and the algorithms used, please refer to the component README. | ||
|
bradleycamacho marked this conversation as resolved.
Outdated
|
||
|
|
||
| Security monitors: Those tools monitor other aspects of several contracts used in optimism: | ||
|
|
||
| * Global Events Monitor: made for to taking YAML rules as configuration and monitoring the events that are emitted on the chain. | ||
|
bradleycamacho marked this conversation as resolved.
Outdated
|
||
| * Liveness Expiration Monitor: monitors the liveness expiration on Safes. | ||
| * Balances Monitor: emits a metric reporting the balances for the configured accounts. | ||
| * Multisig Monitor: The multisig monitor reports the paused status of the OptimismPortal contract. If set, the latest nonce of the configured Safe address. And also if set, the latest presigned nonce stored in One Password. The latest presigned nonce is identified by looking for items in the configured vault that follow a `ready-<nonce>.json` name. The highest nonce of this item name format is reported. | ||
|
bradleycamacho marked this conversation as resolved.
Outdated
|
||
| * Drippie Monitor: tracks the execution and executability of drips within a Drippie contract. | ||
| * Secrets Monitor: takes a Drippie contract as a parameter and monitors for any drips within that contract that use the `CheckSecrets` dripcheck contract. `CheckSecrets` is a dripcheck that allows a drip to begin once a specific secret has been revealed (after a delay period) and cancels the drip if a second secret is revealed. It's important to monitor for these secrets being revealed as this could be a sign that the secret storage platform has been compromised and someone is attempting to exfiltrate the ETH controlled by that drip. | ||
|
bradleycamacho marked this conversation as resolved.
Outdated
|
||
|
|
||
| For more information on these monitors and how to use them, [check out the repo](https://github.com/ethereum-optimism/monitorism?tab=readme-ov-file#monitorism). | ||
|
|
||
| ### `dispute-mon` | ||
|
|
||
| Chain operators should consider running `op-dispute-mon`. It's an incredibly useful securities monitoring service to keep an eye on games, basically giving visibility into all the status of the games for the last 28 days. | ||
|
bradleycamacho marked this conversation as resolved.
Outdated
|
||
|
|
||
| `dispute-mon` is set up and built the same way as `op-challenger`. This means that you can run it the same way (run `make op-dispute-mon` in the directory). | ||
|
|
||
| A basic configuration option would look like this: | ||
|
|
||
| ``` | ||
| OP_DISPUTE_MON_LOG_FORMAT=logfmt | ||
| OP_DISPUTE_MON_METRICS_ENABLED=true | ||
| OP_DISPUTE_MON_METRICS_ADDR=0.0.0.0 | ||
| OP_DISPUTE_MON_METRICS_PORT=7300 | ||
|
|
||
| OP_DISPUTE_MON_L1_ETH_RPC=.. | ||
| OP_DISPUTE_MON_ROLLUP_RPC=.. | ||
| OP_DISPUTE_MON_GAME_FACTORY_ADDRESS=.. | ||
|
|
||
| OP_DISPUTE_MON_HONEST_ACTORS=.. | ||
| ``` | ||
|
|
||
| `OP_DISPUTE_MON_HONEST_ACTORS` is a CSV (no spacing) list of addresses that are used for the honest op-challenger instance(s). | ||
|
bradleycamacho marked this conversation as resolved.
Outdated
|
||
|
|
||
| Additional flags: | ||
|
|
||
| * `OP_DISPUTE_MON_GAME_WINDOW`: This is the window of time to report on games. It should leave a buffer beyond the max game duration for bond claiming. If Fault Proof game parameters are not changes (e.g. MAX\_CLOCK\_DURATION), it is recommended to leave this as the default. | ||
| * `OP_DISPUTE_MON_MONITOR_INTERVAL`: The interval at which to check for new games. Defaults to 30 seconds currently. | ||
| * `OP_DISPUTE_MON_MAX_CONCURRENCY`: The max thread count. Defaults to 5 currently. | ||
|
|
||
| You can find more info on `op-dispute-mon` on [the repo](https://github.com/ethereum-optimism/optimism/tree/develop/op-dispute-mon). | ||
|
|
||
| Chain operators can easily create their grafana dashboard for Dispute Monitor using the following json file: [Download the Dispute Monitor JSON](/grafana/dispute-monitor-1718214549035.json). | ||
|
|
||
| ## Offchain Component Monitoring | ||
|
|
||
| Offchain monitoring allows chain operators to monitor the operation and behavior of nodes and other offchain components. Some of the more common components that you'll likely want to monitor include `op-node`, `op-geth`, `op-proposer`, `op-batcher`, and `op-challenger`. | ||
| The general steps for enabling offchain monitoring is pretty consistent for all the OP components: | ||
|
|
||
| 1. Expose the monitoring port by enabling the `-—metrics.enabled` flag | ||
| 2. Customize the metrics port and address via the `--metrics.port` and `--metrics.addr` flags, respectively | ||
| 3. Use [Prometheus](https://prometheus.io/) to scrape data from the metrics port | ||
| 4. Save the data in `influxdb` | ||
| 5. Share the data with [Grafana](https://grafana.com/) to build your custom dashboard | ||
|
|
||
| ### `op-node` | ||
|
|
||
| `op-node` metrics and monitoring is detailed in the [Node Metrics and Monitoring](/builders/node-operators/management/metrics) guide. To enable metrics, pass the `--metrics.enabled` flag to `op-node` and follow the steps above for customization options. | ||
| See [this curated list](/builders/node-operators/management/metrics#important-metrics) for important metrics to track specifically for `op-node`. | ||
|
|
||
| ### `op-geth` | ||
|
|
||
| To enable metrics, pass the `--metrics.enabled` flag to the op-geth. You can customize the metrics port and address via the `--metrics.port` and `--metrics.addr` flags, respectively. | ||
|
|
||
| ### `op-proposer` | ||
|
|
||
| To enable metrics, pass the `--metrics.enabled` flag to the op-proposer. You can customize the metrics port and address via the `--metrics.port` and `--metrics.addr` flags, respectively. | ||
|
|
||
| You can find more information about these flags in our [Proposer configuration doc](https://docs.optimism.io/builders/chain-operators/configuration/proposer#metricsenabled). | ||
|
|
||
| ### `op-batcher` | ||
|
|
||
| To enable metrics, pass the `--metrics.enabled` flag to the op-batcher. You can customize the metrics port and address via the `--metrics.port` and `--metrics.addr` flags, respectively. | ||
|
|
||
| You can find more information about these flags in our [Batcher configuration doc](https://docs.optimism.io/builders/chain-operators/configuration/batcher#metricsenabled). | ||
|
|
||
| ### `op-challenger` | ||
|
|
||
| The `op-challenger` operates as the *honest actor* in the fault dispute system and defends the chain by securing the `OptimismPortal` and ensuring the game always resolves to the correct state of the chain. | ||
| For verifying the legitimacy of claims, `op-challenger` relies on a synced, trusted rollup node as well as a trace provider (e.g., [Cannon](/stack/protocol/fault-proofs/cannon)). See the [OP-Challenger Explainer](/stack/protocol/fault-proofs/challenger) for more information on this service. | ||
|
|
||
| To enable metrics, pass the `--metrics.enabled` flag to `op-challenger` and follow the steps above for customization options. | ||
|
|
||
| ``` | ||
| --metrics.addr value (default: "0.0.0.0") ($OP_CHALLENGER_METRICS_ADDR) | ||
| Metrics listening address | ||
|
|
||
| --metrics.enabled (default: false) ($OP_CHALLENGER_METRICS_ENABLED) | ||
| Enable the metrics server | ||
|
|
||
| --metrics.port value (default: 7300) ($OP_CHALLENGER_METRICS_PORT) | ||
| Metrics listening port | ||
| ``` | ||
|
|
||
| ## Next Steps | ||
|
|
||
| * If you experience difficulty at any stage of this process, please reach out to [developer support](https://github.com/ethereum-optimism/developers/discussions). | ||
|
bradleycamacho marked this conversation as resolved.
Outdated
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.