Expand Circuit Breaker Troubleshooting

troubleshoot/elasticsearch/circuit-breaker-errors.md

@@ -24,7 +24,7 @@
 
 **Error messages**
 
-If a request triggers a circuit breaker, {{es}} returns an error with a `429` HTTP status code.
+A circuit breaker trips to prevent a request from executing in order to protect the node's stability. When a request triggers a circuit breaker, {{es}} [rejects the request](/troubleshoot/elasticsearch/rejected-requests.md) with a `429` HTTP status code error.
 
 ```js
 {
@@ -45,6 +45,37 @@
 Caused by: org.elasticsearch.common.breaker.CircuitBreakingException: [parent] Data too large, data for [<transport_request>] would be [num/numGB], which is larger than the limit of [num/numGB], usages [request=0/0b, fielddata=num/numKB, in_flight_requests=num/numGB, accounting=num/numGB]
 ```
 
+**Check circuit breaker statistics**
+
+To get statistics about the circuit breaker per node, use one of the following:
+
+* You can use the [get node statistics](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-stats) API:
+
+    ```console
+    GET _nodes/stats?filter_path=nodes.*.breakers
+    ```
+
+    The response provides the following information:
+    - Estimated memory used for the operation.
+    - Memory limit for the circuit breaker.
+    - Total number of times the circuit breaker has been triggered and prevented an out of memory error since node uptime.
+    - And an overhead which is a constant that all estimates for the circuit breaker are multiplied with to calculate a final estimate.
+
+
+* {applies_to}`stack: ga 9.3` Starting with {{es}} version 9.3, you can use the [get circuit breakers statistics](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-circuit-breaker) API:
+
+    ```console
+    GET /_cat/circuit_breaker/
+    ```
+
+    This API provides a concise, human-readable tabular output that is useful for quick monitoring and troubleshooting. The response includes the following information for each circuit breaker:
+    - The circuit breaker name (for example, `request`, `fielddata`, `in_flight_requests`).
+    - The node ID where the circuit breaker is located.
+    - Estimated memory currently in use by the circuit breaker.
+    - Memory limit configured for the circuit breaker.
+    - Total number of times the circuit breaker has been triggered.
+    - Overhead factor applied to memory estimates.
+
 **Check JVM memory usage**
 
 If you’ve enabled Stack Monitoring, you can view JVM memory usage in {{kib}}. In the main menu, click **Stack Monitoring**. On the Stack Monitoring **Overview** page, click **Nodes**. The **JVM Heap** column lists the current memory usage for each node.
@@ -79,3 +110,13 @@
 ```console
 POST _cache/clear?fielddata=true
 ```
+
+## Memory evaluation
+
+Circuit breakers may either directly evaluate memory usage estimates or indirectly limit operations that are likely to cause excessive memory consumption. For example, the `script` circuit breaker checks memory indirectly by rate-limiting Painless/Mustache script compilations. However, even with circuit breakers in place, nodes can still encounter out-of-memory (OOM) conditions. This can occur, for example, because:
+
+- Circuit breaker relies on point-in-time memory usage estimations.
+- Parallel operations may still heap DOS-attack the node even with `parent` circuit breakers.
+- Certain dynamic operations can quickly consume substantial memory. For example [aggregations](/explore-analyze/query-filter/aggregations.md) and [complex queries](elasticsearch://reference/query-languages/query-dsl/compound-queries.md).
+Circuit breakers protect the node's JVM heap. OOM can still trigger due to non-heap memory, for example within the compilation or thread stacks.
+