Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/MigrationGuide.md
Original file line number Diff line number Diff line change
Expand Up @@ -817,6 +817,14 @@ limits control how many requests may wait, execute, or remain parked, and how
much request data they may retain. A request refused at that stage receives
`BadServerTooBusy`, even if it fitted in a single transport chunk.

`MaxQueuedRequestCount` keeps its 1.5.x meaning: established Sessions can have
that many decoded requests waiting. Balanced and TrustedReservations add their
reserved queue entries on top of it. Earlier 2.0 previews carved them out of it,
so a client that kept exactly `MaxQueuedRequestCount` requests outstanding could
receive `BadServerTooBusy`. A server that relied on the smaller queue should
lower `MaxQueuedRequestCount` or set `ResourceIsolationStageOptions.Capacity` for
the RequestQueue stage. See [request and caller limits](ResourceIsolation.md#configuring-request-and-caller-limits).

For workloads with many simultaneous large requests, set
`WithChunkReassemblyBudget(maxBytes)` on the Dependency Injection (DI) server builder or assign
`ServerBase.ChunkReassemblyBudget` before startup. A host opening listeners
Expand Down
14 changes: 14 additions & 0 deletions docs/NodeManagers.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,8 @@ The master node manager builds a routing table keyed by namespace index. During

Multiple managers can serve the same namespace. `RegisterNamespaceManager(string namespaceUri, IAsyncNodeManager nodeManager)` appends a manager to the namespace route instead of replacing the existing route; the routing table also preserves manager order during lifecycle replacement. This is important for namespace 0 and for generated or runtime models that add nodes in namespaces already used by another manager.

Read and Write call `ReadAsync`/`WriteAsync` only on the managers that appear in the route of a requested namespace, in dispatch order, so a request does not pay for managers that cannot own any of its nodes. If any valid node is in a namespace without a route, every manager is called as before. A manager that processes nodes outside the namespaces it registers must register them with `RegisterNamespaceManager` to keep receiving those Reads and Writes.

The master node manager also merges references across managers. Each manager receives the shared `externalReferences` table while `CreateAddressSpaceAsync` runs. After all managers have created their nodes and historical-access advertisement has been reconciled, the master calls `AddReferencesAsync(externalReferences, ...)` on every manager. `MasterNodeManager.CreateExternalReference` is the helper most custom managers use to put a cross-manager reference in that table. The target manager then materialises the reference on the node it owns, so a node owned by one manager can be browseable from a parent, folder, metadata object, or notifier owned by another manager. Attaching a child only in the source manager is not enough when the source and target owners differ.

At runtime, the same reference handling is used for lifecycle-managed managers. A prepared manager builds its address space while hidden from client routing, the master adds its external references during commit, and rollback/removal removes the cross-manager references that were added for that generation.
Expand Down Expand Up @@ -386,6 +388,18 @@ support filter validation, event subscriptions, condition refresh, and role chec
See [monitoring selection](AsyncServerSupport.md#monitored-item-manager-selection)
for the tradeoff between periodic sampling and change notifications.

A sampling group checks the Read permission of an item through
`ValidateRolePermissionsAsync` when it starts sampling the item, and reuses the
verdict until one of its inputs changes: the Session or its effective identity
(ActivateSession with another user, or granted roles refreshed after a role
change), the item's node handle, the `RolePermissions` or `UserRolePermissions`
of any `NodeState`, or the default permissions of a namespace. A revoked
permission therefore yields `BadUserAccessDenied` within one or two sampling
cycles, as before. Each item
is also revalidated at least once per second, which bounds the delay for
permissions a node manager computes without such a change, for example through
`OnReadRolePermissions`.

#### Sampling interval revision

When a client creates or modifies a monitored item the node manager revises the
Expand Down
21 changes: 17 additions & 4 deletions docs/ResourceIsolation.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ These terms describe resource accounting, not permission to access OPC UA data.
| Isolation refusal | A rejection by the resource-isolation provider, for example because capacity or a per-caller limit is exhausted. A rate limiter or normal service authorization can reject work independently. |
| Owner | The accounting group charged for work: an observed address, a verified application or Session identity, or an explicitly configured group. It is not necessarily one person or one client process. |
| Stage | One separately counted resource, such as connections, queued requests, or retained bytes. The API names are listed under [Accounting and lifetime](#accounting-and-lifetime). |
| Floor / reservation | Capacity kept for a particular class of work or configured owner, inside the existing total. Other classes cannot borrow it, even when it is unused. |
| Floor / reservation | Capacity kept for a particular class of work or configured owner. Other classes cannot borrow it, even when it is unused. Floors are inside the existing total, except at the RequestQueue, RequestQueueBytes and ParkedRequest stages, where the default total adds them on top of the configured queue (see [Configuring request and caller limits](#configuring-request-and-caller-limits)). |
| Shared capacity / work-conserving | Capacity available to all classes. If only one owner has work, it can use the unused shared capacity up to its hard limit. The server does not divide it into fixed equal shares. Reserved capacity is excluded. |
| Lease | The provider's record of capacity in use. Disposing the lease releases that charge. It is not a time-limited Session or credential. |
| Bootstrap / reconnect | Initial startup or recovery of a connection or Session. Protected access requires a trusted ingress mapping or, where applicable, an authoritative live Session binding. A request's name alone is not evidence. |
Expand Down Expand Up @@ -323,7 +323,9 @@ Slot totals come from `MaxChannelCount` for Connection/Handshake,
(its default applies to nonpositive values), and the existing request-dispatch
limits for queued, executing, and parked work. The dispatcher already uses at
least 100 queue slots and 100 execution slots. The plan mirrors those
minimums. It does not raise configured totals to make reservations fit.
minimums. Apart from the decoded-request queue stages described under
[Configuring request and caller limits](#configuring-request-and-caller-limits),
it does not raise configured totals to make reservations fit.
`Stages[].Capacity` can apply a smaller isolation limit.

For Balanced and TrustedReservations with the default connection rate limiter
Expand Down Expand Up @@ -584,9 +586,20 @@ for another caller.

### Configuring request and caller limits

`MaxQueuedRequestCount` is the number of decoded requests that established
Sessions can have waiting at once. For the RequestQueue, RequestQueueBytes
and ParkedRequest stages, the default total is the configured limit plus the
bootstrap, reconnect, control and trusted floors, so the shared capacity equals
the configured value. For example, with `MaxQueuedRequestCount = 10000`, ten
clients can each keep 1,000 requests outstanding. Balanced then sizes the
RequestQueue stage at 10,003 entries. RequestExecution keeps its floors inside
the worker limit, so an idle worker remains available for protected requests.
An explicit `Stages[].Capacity` is still an absolute total that includes the
floors, and it cannot exceed the default total.

`RequestQueueBytes` limits the data retained by decoded requests. The default
total is `MaxMessageSize` multiplied by the combined queued, executing, and
parked request limits. This is deliberately conservative accounting, not a
shared capacity is `MaxMessageSize` multiplied by the combined queued, executing,
and parked request limits. This is deliberately conservative accounting, not a
measurement of CLR heap size. Decoder and operation limits continue to apply.

`MaxTrackedOwners` bounds how many caller groups can have resources charged at
Expand Down
1 change: 1 addition & 0 deletions docs/ServerScalability.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ The following settings size a single node for its hardware (see also the sizing

- **`MaxSessionCount`** caps concurrently open sessions. Size **`MaxChannelCount`** (one channel per session) and **`MaxSubscriptionCount`** at or above the target session count.
- **`MaxRequestThreadCount`** caps concurrent request processing. With `DecoupleHeldPublishRequests` enabled it can be sized for the *active* (non-parked) request concurrency rather than the session count. **`MinRequestThreadCount`** pre-warms the pool so a connect burst is not throttled by thread-pool cold-start.
- **`MaxQueuedRequestCount`** is how many decoded requests established sessions can have waiting for a worker. Size it at or above the total pipelining of your clients (for example 10 clients with 1,000 outstanding requests each need 10,000). The reserved queue entries of the Balanced and TrustedReservations isolation profiles are added on top; see [request and caller limits](ResourceIsolation.md#configuring-request-and-caller-limits).
- **`DecoupleHeldPublishRequests`** (default `true`) releases a held `Publish`'s request-processing worker while it waits, so a small worker pool can hold many outstanding long-polls.
- **`MaxMonitoredItemCount`** and **`MaxMonitoredItemsPerSubscription`** (default `0`, no limit) cap the monitored items in the server and in one subscription. `CreateMonitoredItems` answers `Bad_TooManyMonitoredItems` for the items over the limit, and the values are published in `ServerCapabilities.MaxMonitoredItems` and `MaxMonitoredItemsPerSubscription`. Set them on servers reachable by untrusted clients; `MaxMonitoredItemsPerCall` only bounds a single request.
- **`MaxFailedAuthenticationAttempts`** (default 5; `0` disables) is the per-certificate brute-force lockout. A single-certificate client that opens many sessions can trip it on transient handshake failures and then be rejected with `BadUserAccessDenied`; raise or disable it for bulk-connect clients.
Expand Down
13 changes: 9 additions & 4 deletions src/Opc.Ua.Core/Stack/Server/FairRequestQueue.cs
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,11 @@ namespace Opc.Ua
/// admission and non-borrowable partitions; this queue owns ordering and lease lifetimes.
/// </summary>
/// <remarks>
/// The provider's RequestQueue capacity must not exceed the physical maxQueuedRequests.
/// Every queued entry holds a provider lease, so matching capacity envelopes preserve
/// protected slots even when the shared partition is full.
/// The provider's RequestQueue capacity must not exceed the physical bound. A provider that
/// reports its capacity raises the bound from maxQueuedRequests to that capacity, because the
/// default plan adds protected floors on top of the configured queue size. Every queued entry
/// holds a provider lease, so matching capacity envelopes preserve protected slots even when
/// the shared partition is full.
/// </remarks>
internal sealed class FairRequestQueue : IDisposable
{
Expand All @@ -67,7 +69,10 @@ public FairRequestQueue(
{
throw new ArgumentOutOfRangeException(nameof(requestCost));
}
m_maxQueuedRequests = maxQueuedRequests;
m_maxQueuedRequests = provider is IResourceIsolationCapacitySource capacitySource
? (int)Math.Min(int.MaxValue, Math.Max(
maxQueuedRequests, capacitySource.GetStageCapacity(ResourceIsolationStage.RequestQueue)))
: maxQueuedRequests;
m_requestCost = requestCost;
m_useFairScheduling = provider.UseFairScheduling;
m_decoupleHeldPublishRequests = decoupleHeldPublishRequests;
Expand Down
12 changes: 12 additions & 0 deletions src/Opc.Ua.Core/Stack/Server/IServerResourceIsolationProvider.cs
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,18 @@ StatusCode GetRevalidationStatus(
bool controlRequest = false);
}

/// <summary>
/// Reports a stages aggregate capacity including protected floors, so the request queue can
/// size its physical bound to match the providers envelope.
Comment thread
Copilot marked this conversation as resolved.
Resolved
/// </summary>
internal interface IResourceIsolationCapacitySource
{
/// <summary>
/// Gets the aggregate capacity of a stage, including every reserved floor.
/// </summary>
long GetStageCapacity(ResourceIsolationStage stage);
}

/// <summary>
/// Optional capability exposed by server transport callbacks.
/// </summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -878,7 +878,7 @@ private static int IndexOf(
/// Gets whether two entries denote the same NodeManager. Two adapters that wrap the same
/// synchronous NodeManager count as the same NodeManager.
/// </summary>
private static bool AreSameManager(
internal static bool AreSameManager(
IAsyncNodeManager left,
IAsyncNodeManager right)
{
Expand Down
Loading
Loading