Skip to content

docs: add request size limitation#782

Open
pellared wants to merge 13 commits intoopen-telemetry:mainfrom
pellared:content-size-server
Open

docs: add request size limitation#782
pellared wants to merge 13 commits intoopen-telemetry:mainfrom
pellared:content-size-server

Conversation

@pellared
Copy link
Copy Markdown
Member

@pellared pellared commented Mar 31, 2026

Per #781 (comment)

Add request size limitation for HTTP body and gRPC messages to mitigate memory usage risks.

Reference: https://cwe.mitre.org/data/definitions/789.html

The values are taken from otlpreceiver defaults #782 (comment)

  • gRPC - 4 MiB 32 MiB
  • HTTP - 20 MiB 32 MiB

OTel implementations:

@pellared pellared marked this pull request as ready for review March 31, 2026 10:54
@pellared pellared requested a review from Copilot March 31, 2026 10:57
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the OTLP specification documentation to define request size/body limits for OTLP/gRPC and OTLP/HTTP, aiming to reduce memory-exhaustion risk from oversized payloads.

Changes:

  • Documented an OTLP/gRPC request message size limit recommendation (4 MiB) and client behavior on oversize errors.
  • Documented an OTLP/HTTP request body size limit recommendation (20 MiB) and recommended HTTP 413 handling semantics.
  • Added an Unreleased changelog entry for the documentation update.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
docs/specification.md Adds normative guidance for gRPC request message size limits and HTTP request body limits.
CHANGELOG.md Records the docs change in the Unreleased “Added” section.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

pellared and others added 2 commits March 31, 2026 20:45
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@pellared pellared changed the title docs: add request body size limitation docs: add body size limitation Mar 31, 2026
@pellared pellared changed the title docs: add body size limitation docs: add response size limitation Mar 31, 2026
@pellared pellared changed the title docs: add response size limitation docs: add request size limitation Mar 31, 2026
@pellared pellared requested a review from Copilot March 31, 2026 18:49
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@pellared pellared marked this pull request as draft April 1, 2026 07:34
@tigrannajaryan
Copy link
Copy Markdown
Member

@open-telemetry/profiling-maintainers do you think 32MiB request size (uncompressed) is large enough that we practically never hit this limit? If it is not large enough is there such number or we think that legitimate profiles can be arbitrarily large? If there is no practical limit for profiles do we feel comfortable with the approach that there is a known default limitation for payloads (e.g. 32MiB) and all profile emitters are expected to split collected profiles to keep individual payloads under this default limit?

@pellared pellared marked this pull request as ready for review April 8, 2026 13:19
@pellared pellared requested a review from felixge April 8, 2026 13:26
@pellared pellared marked this pull request as draft April 8, 2026 15:04
@pellared

This comment was marked as resolved.

Copy link
Copy Markdown
Member

@florianl florianl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you think 32MiB request size (uncompressed) is large enough that we practically never hit this limit? If it is not large enough is there such number or we think that legitimate profiles can be arbitrarily large? If there is no practical limit for profiles do we feel comfortable with the approach that there is a known default limitation for payloads (e.g. 32MiB) and all profile emitters are expected to split collected profiles to keep individual payloads under this default limit?

Since this limit is configurable, environments using larger profiles can still increase it. However, for most environments, 32 MiB appears to be a reasonable default.

@pellared
Copy link
Copy Markdown
Member Author

However, for most environments, 32 MiB appears to be a reasonable default.

Have you done any testing? Do we have some numbers? Could you e.g.. setup the https://github.com/open-telemetry/opentelemetry-ebpf-profiler for all services (including e.g. Prometheus, Jaeger) on https://github.com/open-telemetry/opentelemetry-demo/?

Also waiting on @felixge with (from #782 (comment)):

I'll provide a more in-depth analysis on the data volumes we are seeing in our deployments here ASAP to help making an informed decision.

@tigrannajaryan
Copy link
Copy Markdown
Member

Since this limit is configurable, environments using larger profiles can still increase it.

I think this only works as a strategy if users have control over receiver's configuration, which may not always be the case.

My preference would be one the following approaches, in the order of priority:

  1. Establish a maximum request size that any reasonable production profile cannot possibly exceed. Anything larger than that is to be classified as malicious. If such a maximum number exists and it is not too big to be defined as default max size then we choose this approach. The limit can still be configurable, but it should practically be never required to change this configuration. We also likely don't need to implement sender payload splitting in this case.
  2. If approach 1 is not possible (i.e. no realistic max size is possible to establish), then we choose a reasonable default max size for receivers (e.g. 32MiB). Also require senders by default to split payloads if they exceed a configurable max size. Default max for senders is the same as default max for receivers (or slightly below if splitting cannot be done precisely).

It is important that interoperability works by default, without requiring configuration. Configuration can be used to fine tune performance (e.g. if larger payloads result in better overall compression and memory usage is acceptable).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants