diff --git a/.gitignore b/.gitignore index 8af61fca4..f042202ac 100644 --- a/.gitignore +++ b/.gitignore @@ -15,4 +15,5 @@ out/ # OS X **/.DS_Store -docs/.artifacts \ No newline at end of file +docs/.artifacts +.artifacts \ No newline at end of file diff --git a/docs/docset.yml b/docs/docset.yml index 241533f24..b694695bb 100644 --- a/docs/docset.yml +++ b/docs/docset.yml @@ -2,9 +2,23 @@ project: 'EDOT Java release notes' cross_links: - docs-content - opentelemetry + - elastic-agent + - apm-agent-java toc: - toc: release-notes + - toc: reference/edot-java subs: + motlp: Elastic Cloud Managed OTLP Endpoint + edot: Elastic Distribution of OpenTelemetry + ecloud: "Elastic Cloud" + edot-cf: "EDOT Cloud Forwarder" + ech: "Elastic Cloud Hosted" + ess: "Elasticsearch Service" + ece: "Elastic Cloud Enterprise" + serverless-full: "Elastic Cloud Serverless" agent: "Elastic Agent" agents: "Elastic Agents" - edot: "Elastic Distribution of OpenTelemetry" \ No newline at end of file + stack: "Elastic Stack" + es: "Elasticsearch" + kib: "Kibana" + ls: "Logstash" \ No newline at end of file diff --git a/docs/reference/edot-java/configuration.md b/docs/reference/edot-java/configuration.md new file mode 100644 index 000000000..984c08269 --- /dev/null +++ b/docs/reference/edot-java/configuration.md @@ -0,0 +1,216 @@ +--- +navigation_title: Configuration +description: Learn how to configure the Elastic Distribution of OpenTelemetry (EDOT) Java Agent, including minimal setup, configuration options, and methods like environment variables and system properties. +applies_to: + stack: + serverless: + observability: + product: + edot_java: ga +products: + - id: cloud-serverless + - id: observability + - id: edot-sdk +--- + +# Configure the EDOT Java agent + +The [minimal configuration](#minimal-configuration) section provides a recommended starting point for EDOT Java configuration. + +See [configuration options](#configuration-options) for details on the supported configuration options and [configuration methods](#configuration-methods) for how to provide them. + +:::{note} +[Declarative configuration](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#declarative-configuration) is not supported. Using it deactivates many agent features. +::: + +## Minimal configuration + +This configuration is provided using [environment variables](#environment-variables), other [configuration methods](#configuration-methods) are also supported. + +```sh +# service name: mandatory for integration in UI and correlation +OTEL_SERVICE_NAME=my-service + +# resource attributes: recommended for integration in UI and correlation, can also include service.name +OTEL_RESOURCE_ATTRIBUTES='service.version=1.0,deployment.environment.name=production' + +# exporter endpoint: mandatory if not using a local Collector accessible on http://localhost:4317 +OTEL_EXPORTER_OTLP_ENDPOINT=https://my-otel-collector + +# exporter authentication: mandatory if endpoint requires authentication +OTEL_EXPORTER_OTLP_HEADERS='Authorization=ApiKey mySecretApiKey' +``` + +For authentication, the `OTEL_EXPORTER_OTLP_HEADERS` can also be used with an APM secret token: +```sh +OTEL_EXPORTER_OTLP_HEADERS='Authorization=Bearer mySecretToken' +``` + +## Configuration options + +EDOT Java instrumentation agent is based on OpenTelemetry Java [SDK](https://github.com/open-telemetry/opentelemetry-java) and [Instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation), and thus supports the following +configuration options: +- [OpenTelemetry Java instrumentation configuration options](https://opentelemetry.io/docs/zero-code/java/agent/configuration/) +- [OpenTelemetry Java SDK configuration options](https://opentelemetry.io/docs/languages/java/configuration/) + +EDOT Java uses different defaults than the OpenTelemetry Java instrumentation for the following configuration options: + +| Option | EDOT Java default | OpenTelemetry Java agent default | EDOT Java version | +|----------------------------------------------------------------------|-------------------|----------------------------------------------------------------------------------------------------------------------------------------------|-------------------| +| `OTEL_RESOURCE_PROVIDERS_AWS_ENABLED` | `true` | `false` ([docs](https://opentelemetry.io/docs/zero-code/java/agent/configuration/#enable-resource-providers-that-are-disabled-by-default)) | 1.0.0+ | +| `OTEL_RESOURCE_PROVIDERS_GCP_ENABLED` | `true` | `false` ([docs](https://opentelemetry.io/docs/zero-code/java/agent/configuration/#enable-resource-providers-that-are-disabled-by-default)) | 1.0.0+ | +| `OTEL_RESOURCE_PROVIDERS_AZURE_ENABLED` | `true` | `false` ([docs](https://opentelemetry.io/docs/zero-code/java/agent/configuration/#enable-resource-providers-that-are-disabled-by-default)) | 1.4.0+ | +| `OTEL_INSTRUMENTATION_RUNTIME-TELEMETRY_EMIT-EXPERIMENTAL-TELEMETRY` | `true` | `false` ([docs](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/runtime-telemetry/README.md)) | 1.4.0+ | +| `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE` | `delta` (*) | `cumulative` ([docs](https://opentelemetry.io/docs/specs/otel/metrics/sdk_exporters/otlp/#additional-environment-variable-configuration)) | 1.0.0+ | + +(*) default value set to `delta` only if not already explicitly set. + +The EDOT Java instrumentation agent also provides configuration options for each of the [supported features](/reference/edot-java/features.md). +This table only contains minimal configuration, see each respective feature for exhaustive configuration options documentation. + +| Option | Default | Feature | EDOT Java version | +|--------------------------------------------------------|---------|--------------------------------------------------------------------------------------------------------------------------|-------------------| +| `OTEL_INFERRED_SPANS_ENABLED` | `false` | [Inferred spans](/reference/edot-java/features.md#inferred-spans) | 1.0.0+ | +| `OTEL_JAVA_EXPERIMENTAL_SPAN_STACKTRACE_MIN_DURATION` | `5ms` | [Span stacktrace](/reference/edot-java/features.md#span-stacktrace) | 1.0.0+ | +| `ELASTIC_OTEL_UNIVERSAL_PROFILING_INTEGRATION_ENABLED` | `auto` | [Elastic Universal profiling integration](/reference/edot-java/features.md#elastic-universal-profiling-integration) | 1.0.0+ | +| `OTEL_INSTRUMENTATION_OPENAI_CLIENT_ENABLED` | `true` | [OpenAI client instrumentation](/reference/edot-java/supported-technologies.md#openai-client-instrumentation) | 1.4.0+ | +| `ELASTIC_OTEL_JAVAAGENT_LOG_LEVEL` | `INFO` | [Agent logging](#agent-logging) | 1.5.0+ | +| `ELASTIC_OTEL_VERIFY_SERVER_CERT` | `true` | [Exporter certificate verification](#exporter-certificate-verification) | 1.5.0+ | + +## OpenAI Client settings + +The following settings are available: + +| Option | default | description | +|-------------------------------------------------------|---------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `OTEL_INSTRUMENTATION_OPENAI_CLIENT_ENABLED` | `true` | enables or disable OpenAI instrumentation | +| `ELASTIC_OTEL_JAVA_INSTRUMENTATION_GENAI_EMIT_EVENTS` | value of `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` | If set to `true`, the agent will generate log events for OpenAI requests and responses. Potentially sensitive content will only be included if `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` is `true` | +| `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` | `false` | If set to `true`, enables the capturing of OpenAI request and response content in the log events outputted by the agent. ↪ | + +## Central configuration + +```{applies_to} +serverless: unavailable +stack: preview 9.1 +product: + edot_java: preview 1.5.0 +``` + +APM Agent Central Configuration lets you configure EDOT Java instances remotely, see [Central configuration docs](opentelemetry://reference/central-configuration.md) for more details. + +### Turn on central configuration + +To activate central configuration, set the `ELASTIC_OTEL_OPAMP_ENDPOINT` environment variable to the OpAMP server endpoint. + +```sh +export ELASTIC_OTEL_OPAMP_ENDPOINT=http://localhost:4320/v1/opamp +``` + +To deactivate central configuration, remove the `ELASTIC_OTEL_OPAMP_ENDPOINT` environment variable and restart the instrumented application. + +### Central configuration settings + +You can modify the following settings for EDOT Java through APM Agent Central Configuration: + +| Setting | Central configuration name | Type | +|---------|--------------------------|------| +| Logging level | `logging_level` | Dynamic | +| Turn off instrumentations | `deactivate_instrumentations` | Dynamic | +| Turn off all instrumentations | `deactivate_all_instrumentations` | Dynamic | +| Send traces | `send_traces` | Dynamic | +| Send metrics | `send_metrics` | Dynamic | +| Send logs | `send_logs` | Dynamic | + +Dynamic settings can be changed without having to restart the application. + +## Configuration methods + +Configuration can be provided through multiple [configuration methods](#configuration-methods): + +* [Environment variables](#environment-variables) +* [System properties](#system-properties) +* [Properties configuration file](#properties-configuration-file) + +Configuration options are applied with the following priorities: + +- [environment variables](#system-properties) take precedence over [system properties](#system-properties) and [properties configuration file](#properties-configuration-file). +- [system properties](#system-properties) take precedence on [properties configuration file](#properties-configuration-file). + +:::{important} +[Declarative configuration](https://opentelemetry.io/docs/specs/otel/configuration/#declarative-configuration) is not supported. +::: + +### Environment variables + +Environment variables provide a cross-platform way to configure EDOT Java and is especially useful in containerized environments. + +Define environment variables before starting the JVM: + +```sh +export OTEL_SERVICE_NAME=my-service +java ... +``` + +### System properties + +These configuration options can be seen by anything that can see the executed command-line. + +Define system properties at the JVM start, usually on the command-line: + +```sh +java -Dotel.service.name=my-service ... +``` + +When modifying the JVM command line options is not possible, using the `JAVA_TOOL_OPTIONS` environment variable could +be used to provide the system properties, for example: + +```sh +export JAVA_TOOL_OPTIONS='-Dotel.service.name=my-service' +``` + +### Properties configuration file + +EDOT Java can be configured using a java properties configuration file. + +Before starting the JVM, create and populate the configuration file and specify where to find it: + +```sh +echo otel.service.name=my-service > my.properties +java -Dotel.javaagent.configuration-file=my.properties ... +``` + +## Agent logging + +The EDOT Java agent provides the ability to control the agent log verbosity by setting the log level with the `ELASTIC_OTEL_JAVAAGENT_LOG_LEVEL` configuration option (`INFO` by default). + +The following log levels are supported: `TRACE`, `DEBUG`, `INFO`, `WARN`, `ERROR`, `FATAL` and `OFF`. + +For [troubleshooting](docs-content://troubleshoot/ingest/opentelemetry/edot-sdks/java/index.md#agent-debug-logging), the `ELASTIC_OTEL_JAVAAGENT_LOG_LEVEL=DEBUG` is a recommended alternative to `OTEL_JAVAAGENT_DEBUG=true` as it provides span information in JSON format. + +This feature relies on the `OTEL_JAVAAGENT_LOGGING` configuration option to be set to `elastic` (default), the `simple` value from contrib is not supported. + +Setting `OTEL_JAVAAGENT_LOGGING=none` or `ELASTIC_OTEL_JAVAAGENT_LOG_LEVEL=OFF` disables agent logging feature. + +Setting `OTEL_JAVAAGENT_LOGGING=application` will disable EDOT agent logging feature and attempt to use the application logger. +As [documented here in the contrib documentation](https://opentelemetry.io/docs/zero-code/java/agent/configuration/#java-agent-logging-output), +support for this depends on the application and logging libraries used. + +## Exporter certificate verification + +The EDOT Java agent provides the ability to toggle the exporter endpoint certificate verification with the `ELASTIC_OTEL_VERIFY_SERVER_CERT` configuration option (`true` by default). + +When the endpoint certificate is not trusted by the JVM where the agent runs, the common symptom is security-related exceptions with the following message: `unable to find valid certification path to requested target`. + +This is common in the following scenarios: +- When endpoint uses a self-signed certificate not trusted by the JVM +- When the certificate authority used by the endpoint certificate is not trusted by the JVM + +One solution is to add the certificate or certificate authority to the JVM trust store, which requires modifying the JVM trust store. + +If trust store modification is not possible or not practical, for example when troubleshooting or working with a local deployment, certificate verification can be disabled by setting `ELASTIC_OTEL_VERIFY_SERVER_CERT` to `false`. This however need to be evaluated carefully as it lowers the communication security and could allow for man-in-the-middle attacks where the data could be intercepted between the agent and the collector endpoint. + +## Prevent logs export + +To prevent logs from being exported, set `OTEL_LOGS_EXPORTER` to `none`. However, application logs might still be gathered and exported by the Collector through the `filelog` receiver. + +To prevent application logs from being collected and exported by the Collector, refer to [Exclude paths from logs collection](elastic-agent://reference/edot-collector/config/configure-logs-collection.md#exclude-logs-paths). \ No newline at end of file diff --git a/docs/reference/edot-java/features.md b/docs/reference/edot-java/features.md new file mode 100644 index 000000000..2068cb1f9 --- /dev/null +++ b/docs/reference/edot-java/features.md @@ -0,0 +1,81 @@ +--- +navigation_title: Features +description: Explore the features of the Elastic Distribution of OpenTelemetry (EDOT) Java Agent, including inherited OpenTelemetry features and exclusive Elastic enhancements like inferred spans and universal profiling integration. +applies_to: + stack: + serverless: + observability: + product: + edot_java: ga +products: + - id: cloud-serverless + - id: observability + - id: edot-sdk +--- + +# Features of the EDOT Java Agent + +The EDOT Java agent is a [distribution](https://opentelemetry.io/docs/concepts/distributions/) of [OpenTelemetry Java Instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation) agent. It inherits all the features of the OpenTelemetry Java Instrumentation to capture logs, metrics, and traces. + +The EDOT Java agent also provides: + +- Exclusive features that are not available in the [OpenTelemetry Java Instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation). +- Features of [OpenTelemetry Java Instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation) with [different default configuration](/reference/edot-java/configuration.md#configuration-options). + +In addition to the features listed, refer to [Supported technologies](/reference/edot-java/supported-technologies.md). + +## Resource attributes + +The EDOT Java agent includes the following resource attributes providers from [opentelemetry-java-contrib](https://github.com/open-telemetry/opentelemetry-java-contrib/): + +- AWS: [aws-resources](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/aws-resources). Turned on by default. +- GCP: [gcp-resources](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/gcp-resources). Turned on by default. +- Application server service name detection: [resource-providers](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/resource-providers). + +## Inferred spans + +The EDOT Java agent includes the [Inferred Spans Extension](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/inferred-spans) from [opentelemetry-java-contrib](https://github.com/open-telemetry/opentelemetry-java-contrib/). This extension provides the ability to enhance the traces by creating spans from [async-profiler](https://github.com/async-profiler/async-profiler) data without the need of explicit instrumentation of corresponding spans. + +This feature is turned off by default and can be activated by setting `OTEL_INFERRED_SPANS_ENABLED` to `true`. Refer to [Inferred-spans](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/inferred-spans) documentation for configuration options. + +## Span stacktrace + +The EDOT Java agent includes the [Span Stacktrace Extension](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/span-stacktrace) from [opentelemetry-java-contrib](https://github.com/open-telemetry/opentelemetry-java-contrib/). + +This feature is activated by default and allows to capture a stacktrace for spans that have a duration above a threshold. The `OTEL_JAVA_EXPERIMENTAL_SPAN_STACKTRACE_MIN_DURATION` configuration option, which defaults to `5ms`, allows to configure the minimal duration threshold. A negative value turns off the feature. + +Refer to [span-stacktrace](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/span-stacktrace) for configuration options. + +## Runtime metrics + +Experimental runtime metrics are turned on by default. + +Set `OTEL_INSTRUMENTATION_RUNTIME_TELEMETRY_EMIT_EXPERIMENTAL_TELEMETRY` to `false` to turn them off. + +## Metric temporality + +{{es}} and {{kib}} work best with metrics provided in delta-temporality. Therefore, the EDOT Java changes the default value of `OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE` to `DELTA`. You can override this default if needed, though some provided {{kib}} dashboards will not work correctly if you do it. + +## Central configuration + +You can manage EDOT Java configurations through the [APM Agent Central Configuration feature](docs-content://solutions/observability/apm/apm-agent-central-configuration.md) in the Applications UI. + +Refer to [Central configuration](opentelemetry://reference/central-configuration.md) for more information. + +## Elastic Universal Profiling integration + +[Universal Profiling](https://www.elastic.co/observability/universal-profiling) integration provides the ability to correlate traces with profiling data from the Elastic universal profiler. This feature is turned on by default on supported systems, and turned off otherwise. + +Refer to [universal-profiling-integration](https://github.com/elastic/elastic-otel-java/tree/main/universal-profiling-integration) for details and configuration options. + +## Baggage + +[Baggage](https://opentelemetry.io/docs/concepts/signals/baggage/) provides a key-value store that allows to store +and propagate contextual information to traces, metrics, and logs across services. + +This feature requires minimal code changes for creating and accessing the baggage using the [OpenTelemetry Java API](https://github.com/open-telemetry/opentelemetry-java). Baggage entries can be automatically added to spans and logs through these [configuration](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/baggage-processor#usage-with-sdk-auto-configuration) settings: + +- `OTEL_JAVA_EXPERIMENTAL_SPAN_ATTRIBUTES_COPY_FROM_BAGGAGE_INCLUDE` +- `OTEL_JAVA_EXPERIMENTAL_LOG_ATTRIBUTES_COPY_FROM_BAGGAGE_INCLUDE` + +Refer to [baggage-processor](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/baggage-processor) and the [baggage example](https://github.com/elastic/elastic-otel-java/tree/main/examples/baggage) for more details. diff --git a/docs/reference/edot-java/index.md b/docs/reference/edot-java/index.md new file mode 100644 index 000000000..191841bd2 --- /dev/null +++ b/docs/reference/edot-java/index.md @@ -0,0 +1,38 @@ +--- +navigation_title: EDOT Java +description: Introduction to the Elastic Distribution of OpenTelemetry (EDOT) Java Agent, a customized version of the OpenTelemetry Java agent for capturing traces, metrics, and logs. +applies_to: + stack: + serverless: + observability: + product: + edot_java: ga +products: + - id: cloud-serverless + - id: observability + - id: edot-sdk +--- + +# Elastic Distribution of OpenTelemetry Java + +The {{edot}} (EDOT) Java is a customized version of the [OpenTelemetry Java agent](https://github.com/open-telemetry/opentelemetry-java-instrumentation), configured for the best experience with Elastic Observability. + +Use EDOT Java to start the OpenTelemetry SDK with your Java application, and automatically capture tracing data, performance metrics, and logs. Traces, metrics, and logs can be sent to any OpenTelemetry Protocol (OTLP) Collector you choose. + +A goal of this distribution is to avoid introducing proprietary concepts in addition to those defined by the wider OpenTelemetry community. For any additional features introduced, Elastic aims at contributing them back to the OpenTelemetry project. + +## Features + +In addition to all the features of OpenTelemetry Java, with EDOT Java you have access to the following: + +* Improvements and bug fixes contributed by the Elastic team before the changes are available in OpenTelemetry repositories. +* Optional features that can enhance OpenTelemetry data that is being sent to Elastic. +* Elastic-specific processors that ensure optimal compatibility when exporting OpenTelemetry signal data to an Elastic backend like an Elastic Observability deployment. +* Preconfigured collection of tracing and metrics signals, applying some opinionated defaults, such as which sources are collected by default. +* Compatibility with APM Agent Central Configuration to modify the settings of the EDOT Java agent without having to restart the application. + +Follow the step-by-step instructions in [Setup](/reference/edot-java/setup/index.md) to get started. + +## Release notes + +For the latest release notes, including known issues, deprecations, and breaking changes, refer to [EDOT Java release notes](/release-notes/index.md) \ No newline at end of file diff --git a/docs/reference/edot-java/migration.md b/docs/reference/edot-java/migration.md new file mode 100644 index 000000000..c6ab5412f --- /dev/null +++ b/docs/reference/edot-java/migration.md @@ -0,0 +1,228 @@ +--- +navigation_title: Migration +description: Migrate from the Elastic APM Java agent to the Elastic Distribution of OpenTelemetry Java (EDOT Java). +applies_to: + stack: + serverless: + observability: + product: + edot_java: ga +products: + - id: cloud-serverless + - id: observability + - id: edot-sdk + - id: apm-agent +--- + +# Migrate to EDOT Java from the Elastic APM Java agent + +Compared to the Elastic APM Java agent, the {{edot}} Java presents a number of advantages: + +- Fully automatic instrumentation with zero code changes. No need to modify application code. +- Capture, send, transform, and store data in an OpenTelemetry native way. This includes for example the ability to use all features of the OpenTelemetry SDK for manual tracing, data following semantic conventions, or ability to use intermediate collectors and processors. +- OpenTelemetry Java Instrumentation provides a [broad coverage of libraries, frameworks, and applications](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md). +- EDOT Java is built on top of OpenTelemetry SDK and conventions, ensuring compatibility with community tools, vendor-neutral backends, and so on. + +## Migration steps + +Follow these steps to migrate from the legacy Elastic APM Java agent to the {{edot}} Java. + +::::::{stepper} + +::::{step} (Optional) Migrate manual instrumentation API + +Migrate usages of the [Elastic APM Agent API](apm-agent-java://reference/public-api.md) to OpenTelemetry API: + +- For [Annotation API](apm-agent-java://reference/public-api.md#api-annotation), refer to [OpenTelemetry Annotations](https://opentelemetry.io/docs/zero-code/java/agent/annotations/). +- For [Transaction API](apm-agent-java://reference/public-api.md#api-transaction), refer to [OpenTelemetry API](https://opentelemetry.io/docs/zero-code/java/agent/api/). + +:::{note} +Migration of application code using these APIs and annotations is not strictly required when deploying the EDOT agent. If not migrated, spans, transactions, and metrics that were previously created with those custom API calls and annotations will no longer be generated. OpenTelemetry instrumentation coverage might replace the need for some or all of these custom code changes. +::: +:::: + +::::{step} Replace configuration options +Refer to the [Configuration mapping](#configuration-mapping). Refer to [Configuration](/reference/edot-java/configuration.md) for ways to provide configuration settings. +:::: + +::::{step} Replace the agent binary + +Remove the `-javaagent:` argument that contains the Elastic APM Java agent from the JVM arguments. Then add the `-javaagent:` argument to the JVM arguments to use EDOT Java, and restart the application or follow [Kubernetes instructions](/reference/edot-java/setup/k8s.md) or [Runtime attach instructions](/reference/edot-java/setup/runtime-attach.md) if applicable. Refer to [Setup](/reference/edot-java/setup/index.md). +:::: + +:::::: + +## Configuration mapping + +The following are Elastic APM Java agent settings that you can migrate to EDOT Java. + +### `server_url` + +The Elastic [`server_url`](apm-agent-java://reference/config-reporter.md#config-server-url) option corresponds to the OpenTelemetry [`OTEL_EXPORTER_OTLP_ENDPOINT`](https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/#otel_exporter_otlp_endpoint) option. + +### `server_urls` + +The Elastic [`server_urls`](apm-agent-java://reference/config-reporter.md#config-server-urls) option has no equivalent OpenTelemetry option. You can only specify one endpoint. + +Use [OTEL_EXPORTER_OTLP_ENDPOINT](https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/#otel_exporter_otlp_endpoint) instead. + +### `secret_token` + +The Elastic [`secret_token`](apm-agent-java://reference/config-reporter.md#config-secret-token) option corresponds to the OpenTelemetry [OTEL_EXPORTER_OTLP_HEADERS](https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/#otel_exporter_otlp_headers) option. + +For example: `OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer an_apm_secret_token"`. + +### `api_key` + +The Elastic [`api_key`](apm-agent-java://reference/config-reporter.md#config-api-key) option corresponds to the OpenTelemetry [OTEL_EXPORTER_OTLP_HEADERS](https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/#otel_exporter_otlp_headers) option. + +For example:`OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey an_api_key"`. + +### `service_name` + +The Elastic [`service_name`](apm-agent-java://reference/config-core.md#config-service-name) option corresponds to the OpenTelemetry [OTEL_SERVICE_NAME](https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/#otel_service_name) option. + +The service name value can also be set using [OTEL_RESOURCE_ATTRIBUTES](https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/#otel_resource_attributes). + +For example: `OTEL_RESOURCE_ATTRIBUTES=service.name=myservice`. If `OTEL_SERVICE_NAME` is set, it takes precedence over the resource attribute. + +### `enabled` + +The Elastic [`enabled`](apm-agent-java://reference/config-core.md#config-enabled) option corresponds to the OpenTelemetry [OTEL_JAVAAGENT_ENABLED](https://opentelemetry.io/docs/zero-code/java/agent/disable/) option. + +### `service_version` + +The Elastic [`service_version`](apm-agent-java://reference/config-core.md#config-service-version) option corresponds to setting the `service.version` key in [OTEL_RESOURCE_ATTRIBUTES](https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/#otel_resource_attributes). + +For example: `OTEL_RESOURCE_ATTRIBUTES=service.version=1.2.3`. + +### `environment` + +The Elastic [`environment`](apm-agent-java://reference/config-core.md#config-environment) option corresponds to setting the `deployment.environment` key in [OTEL_RESOURCE_ATTRIBUTES](https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/#otel_resource_attributes). + +For example: `OTEL_RESOURCE_ATTRIBUTES=deployment.environment=testing`. + +### `global_labels` + +The Elastic [`global_labels`](apm-agent-java://reference/config-core.md#config-global-labels) option corresponds to adding `key=value` comma separated pairs in [OTEL_RESOURCE_ATTRIBUTES](https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/#otel_resource_attributes). + +For example: `OTEL_RESOURCE_ATTRIBUTES=alice=first,bob=second`. Such labels will result in labels.key=value attributes on the server, e.g. labels.alice=first + +### `trace_methods` + +The Elastic [`trace_methods`] option can be replaced by the [`OTEL_INSTRUMENTATION_METHODS_INCLUDE`](https://opentelemetry.io/docs/zero-code/java/agent/annotations/#creating-spans-around-methods-with-otelinstrumentationmethodsinclude) OpenTelemetry option, however the syntax is different and the ability to use wildcards is more limited. + +### `capture_jmx_metrics` + +The Elastic [`capture_jmx_metrics`](apm-agent-java://reference/config-jmx.md#config-capture-jmx-metrics) option can be replaced by +[OpenTelemetry JMX Insight](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/jmx-metrics/javaagent/) feature which is included in EDOT Java. + +The JMX Insight feature provides the following benefits: + +- Ability to define custom metrics using YAML. +- Capturing metrics with pre-defined metrics by using `OTEL_JMX_TARGET_SYSTEM` configuration option. + +### `capture_headers` + +Replace the Elastic `capture_headers` option with the following options: + +- `otel.instrumentation.http.server.capture-request-headers` for HTTP server request +- `otel.instrumentation.http.server.capture-response-headers` for HTTP server response +- `otel.instrumentation.http.client.capture-request-headers` for HTTP client request +- `otel.instrumentation.http.client.capture-response-headers` for HTTP client response +- `otel.instrumentation.messaging.experimental.capture-headers` for messaging + +The `capture_headers` option is dynamically adjustable, while the `otel.*` options are statically set by startup and cannot be subsequently adjusted. + +### `span_stack_trace_min_duration` + +Replace the Elastic `span_stack_trace_min_duration` option with [`OTEL_JAVA_EXPERIMENTAL_SPAN_STACKTRACE_MIN_DURATION`](/reference/edot-java/features.md#span-stacktrace). + +### `disable_instrumentations` + +Replace the `disable_instrumentations` option, which allows to selectively disable instrumentation (opt-out), with `OTEL_INSTRUMENTATION__ENABLED` where `` is the instrumentation name. + +See [OpenTelemetry documentation](https://opentelemetry.io/docs/zero-code/java/agent/disable/) for reference and values. + +### `enable_instrumentations` + +The `enable_instrumentations` option allows to disable all instrumentation enabled by default and selectively enable instrumentation (opt-in) can be replaced with: + +- `OTEL_INSTRUMENTATION_COMMON_DEFAULT_ENABLED` = `false` to disable instrumentations enabled by default. +- `OTEL_INSTRUMENTATION__ENABLED` = `true` where `` is the name of the instrumentation to enable. See [OpenTelemetry documentation](https://opentelemetry.io/docs/zero-code/java/agent/disable/) for reference and values. + +### `hostname` + +The Elastic [`hostname`](apm-agent-java://reference/config-core.md#config-hostname) option corresponds to setting the `host.name` key in [OTEL_RESOURCE_ATTRIBUTES](https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/#otel_resource_attributes). + +For example: `OTEL_RESOURCE_ATTRIBUTES=host.name=myhost`. + +### `service_node_name` + +The Elastic [`service_node_name`](apm-agent-java://reference/config-core.md#config-service-node-name) option corresponds to setting the `service.instance.id` key in [OTEL_RESOURCE_ATTRIBUTES](https://opentelemetry.io/docs/concepts/sdk-configuration/general-sdk-configuration/#otel_resource_attributes). Warning: by default this is a generated unique ID; if you set this it must be a unique value for each JVM otherwise metric views cannot be correctly aggregated nor disambiguated + +For example: `OTEL_RESOURCE_ATTRIBUTES=service.instance.id=myserviceinstance001`. + +### `cloud_provider` + +The Elastic [`cloud_provider`](apm-agent-java://reference/config-core.md#config-cloud-provider) option corresponds to the per-provider `otel.resource.providers.{provider}.enabled` configuration options. + +By default, with EDOT `otel.resource.providers.{provider}.enabled` is set to `true`, this is equivalent to the `cloud_provider` default valuem which is `auto`, or automatically detect cloud providers. Notice that this behavior differs from the contrib OpenTelemetry distribution. + +When the cloud provider is known, or there is none, turning off the non-relevant providers with `otel.resource.providers.{provider}.enabled = false` allows to [minimize the application startup overhead](/reference/edot-java/overhead.md#optimizing-application-startup). + +### `log_sending` + +The Elastic [`log_sending`](apm-agent-java://reference/config-logging.md#config-log-sending) option allows capturing and +sending application logs directly to APM Server without storing them on disk and ingesting them with a separate tool. + +With EDOT, application logs are automatically captured and sent by default. + +This feature is controlled by `otel.logs.exporter`, which is set to `otlp` by default. You can turn it off by setting `otel.logs.exporter` to `none`. + +### `verify_server_cert` + +The Elastic [`verify_server_cert`](apm-agent-java://reference/config-reporter.md#config-verify-server-cert) option allows you to disable server certificate validation. + +With EDOT, the equivalent configuration option is `ELASTIC_OTEL_VERIFY_SERVER_CERT` (default `true`), see [configuration](./configuration.md#exporter-certificate-verification) for details. + +## Limitations + +The following limitations apply to EDOT Java. + +### Supported Java versions + +EDOT Java agent and OpenTelemetry Java instrumentation are only compatible with Java 8 and later. + +### Missing instrumentations + +Support for LDAP client instrumentation is not currently available in EDOT Java. + +### Central and dynamic configuration + +You can manage EDOT Java configurations through the [central configuration feature](docs-content://solutions/observability/apm/apm-agent-central-configuration.md) in the Applications UI. + +Refer to [Central configuration](opentelemetry://reference/central-configuration.md) for more information. + +### Span compression + +EDOT Java does not implement [span compression](docs-content://solutions/observability/apm/spans.md#apm-spans-span-compression). + +### Breakdown metrics + +EDOT Java is not sending metrics that power the [Breakdown metrics](docs-content://solutions/observability/apm/metrics.md#_breakdown_metrics). + +### No remote attach + +There is currently no EDOT Java equivalent for starting the agent with the [remote attach](apm-agent-java://reference/setup-attach-cli.md) capability. The `-javaagent:` option is the preferred startup mechanism. + +A migration path is available for starting the agent with [self attach](apm-agent-java://reference/setup-attach-api.md), which is to use [runtime attachment](/reference/edot-java/setup/runtime-attach.md). Some [limitations](/reference/edot-java/setup/runtime-attach.md#limitations) +apply, and the agent must be started early during application startup. + +### Micrometer turned off by default + +By default, Micrometer instrumentation is inactive and doesn't capture metrics. To turn it on, use the `otel.instrumentation.micrometer.enabled=true` setting. + +## Troubleshooting + +If you're encountering issues during migration, refer to the [EDOT Java troubleshooting guide](docs-content://troubleshoot/ingest/opentelemetry/edot-sdks/java/index.md). \ No newline at end of file diff --git a/docs/reference/edot-java/overhead.md b/docs/reference/edot-java/overhead.md new file mode 100644 index 000000000..93a878645 --- /dev/null +++ b/docs/reference/edot-java/overhead.md @@ -0,0 +1,68 @@ +--- +navigation_title: Performance overhead +description: This page details the expected performance impact when instrumenting Java applications with the Elastic Distribution of OpenTelemetry SDK, including benchmarks and optimization tips. +applies_to: + stack: + serverless: + observability: + product: + edot_java: ga +products: + - id: cloud-serverless + - id: observability + - id: edot-sdk +--- + +# Performance overhead of the EDOT SDK for Java + +This page details the expected performance impact when instrumenting Java applications with the Elastic Distribution of OpenTelemetry SDK, including benchmarks and optimization tips. + +While designed to have minimal performance overhead, the EDOT Java agent, like any instrumentation agent, executes within the application process and thus has a small influence on the application performance. + +This performance overhead depends on the application's technical architecture, its configuration and environment, and the load. These factors are not easy to reproduce on their own, and all applications are different, so it is not possible to provide a simple answer. + +You can measure the performance overhead on the JVM by using the following high-level metrics: + +- Application startup time +- Application response time, which can be directly perceived by users +- CPU usage +- Garbage collector activity: more memory allocation means more GC activity, thus increasing overall CPU usage and potentially reducing application responsiveness +- Memory usage: how much more memory (heap/non-heap) is needed + +## Benchmark + +While Elastic can't provide generically applicable, accurate numbers about the impact on the previous metrics, synthetic benchmarks are executed with a sample application, which allows to provide an estimate and comparison between agents. Those numbers are only provided as indicators: use them as a framework to evaluate and measure the overhead on your applications. + +For example, the application startup overhead going from 5s to 6s (+1s, +20%) does not mean an application having a startup time of 15s will now start in 18s but that you can expect to have about at least one extra second of startup time and the overall impact remains limited. + +The following table compares the classic Elastic APM Java Agent with the EDOT Java Agent and the same benchmark without an agent. + +| | No agent | EDOT Java instrumentation | Elastic APM Java agent | +|------------------------------|-----------|---------------------------|------------------------| +| Startup time | 5.55 s | 6.82 s | 6.87 s | +| Request latency (p95) | 1.96 ms | 2.06 ms | 2.08 ms | +| Total system cpu utilization | 53.82 % | 54.25 % | 56.92 % | +| Total allocated memory | 21.54 gb | 26.37 gb | 22.15 gb | +| Total GC pauses | 106 ms | 123 ms | 120 ms | +| Max heap used | 436.71 mb | 478.46 mb | 573.92 mb | + +The main difference between the two agents is that, unlike EDOT, Elastic APM Java agent recycles in-memory data structures which allows to reduce the overall allocated memory and thus reduces a bit the overhead on the garbage collector. + +This difference is also the reason why we observe a difference in the maximum heap usage as more data structures are kept in-memory when possible and not recycled by the garbage collector. This however does not mean that Elastic APM Java agent requires about 100mb more of memory compared to EDOT, but that when there is no limitation on heap memory usage the agent will use available memory to minimize memory allocation. + +## Optimizing application startup + +With EDOT Java, the following resource attribute providers are enabled by default: + +- AWS: [`OTEL_RESOURCE_PROVIDERS_AWS_ENABLED`](/reference/edot-java/configuration.md#configuration-options) = `true` +- GCP: [`OTEL_RESOURCE_PROVIDERS_GCP_ENABLED`](/reference/edot-java/configuration.md#configuration-options) = `true` +- Azure: [`OTEL_RESOURCE_PROVIDERS_AZURE_ENABLED`](/reference/edot-java/configuration.md#configuration-options) = `true` + +Because those resource attributes providers rely on metadata endpoints, they might require a few HTTP requests. When the cloud provider is known or none is being used, it might be relevant to selectively turn them off by setting their respective configuration options to `false`. + +Also, each activated instrumentation adds instrumentation overhead. You can control this by applying one of the following strategies: + +- [Turn off instrumentations selectively](https://opentelemetry.io/docs/zero-code/java/agent/disable/#suppressing-specific-agent-instrumentation). +- [Turn off all instrumentations and selectively turn on the ones you need](https://opentelemetry.io/docs/zero-code/java/agent/disable/#enable-only-specific-instrumentation). + +Note that some instrumentation relies on other instrumentation to function properly. When selectively turning on instrumentations, make sure to turn on the transitive instrumentation dependencies too. diff --git a/docs/reference/edot-java/setup/index.md b/docs/reference/edot-java/setup/index.md new file mode 100644 index 000000000..babb25017 --- /dev/null +++ b/docs/reference/edot-java/setup/index.md @@ -0,0 +1,99 @@ +--- +navigation_title: Setup +description: Instructions for setting up the Elastic Distribution of OpenTelemetry (EDOT) Java in various environments, including Kubernetes and others. +applies_to: + stack: + serverless: + observability: + product: + edot_java: ga +products: + - id: cloud-serverless + - id: observability + - id: edot-sdk +--- + +# Set up the EDOT Java Agent + +Learn how to set up the {{edot}} (EDOT) Java in various environments, including Kubernetes and others. + +:::{warning} +Avoid using the Java SDK alongside any other APM agent, including Elastic APM agents. Running multiple agents in the same application process may lead to conflicting instrumentation, duplicate telemetry, or other unexpected behavior. +::: + +## Kubernetes + +For Kubernetes, use the OTel Kubernetes Operator. The Operator also manages the auto-instrumentation of Java applications. Follow the [quickstart guide](docs-content://solutions/observability/get-started/opentelemetry/quickstart/index.md) for Kubernetes or learn more about [instrumentation details on Kubernetes for Java](/reference/edot-java/setup/k8s.md). + +## Runtime attach + +For environments where modifying the JVM arguments or configuration is not possible, or when including the EDOT Java in the application binary is necessary or preferred, use the [runtime attach](/reference/edot-java/setup/runtime-attach.md) setup option. + +## All other environments + +Follow the following Java setup guide for all other environments. + +## Download the agent + +You can download the latest release version of the EDOT Java agent from [![Maven Central](https://img.shields.io/maven-central/v/co.elastic.otel/elastic-otel-javaagent?label=elastic-otel-javaagent&style=for-the-badge)](https://mvnrepository.com/artifact/co.elastic.otel/elastic-otel-javaagent/latest) + +## Prerequisites + +Complete the steps in the [Quickstart](docs-content://solutions/observability/get-started/opentelemetry/quickstart/index.md) section that corresponds to your Elastic deployment model. + +## Configure the Java agent + +The minimal configuration to send data involves setting the values for `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_HEADERS` environment variables. + +Set the `service.name` resource attribute explicitly with `OTEL_SERVICE_NAME` as it allows to qualify captured data and group multiple service instances together. + +The following is an example that sets the `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, and `OTEL_SERVICE_NAME` environment variables: + +```sh +export OTEL_EXPORTER_OTLP_ENDPOINT=https://my-deployment.apm.us-west1.gcp.cloud.es.io +export OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey P....l" +export OTEL_SERVICE_NAME="my-awesome-service" +``` + +For more advanced configuration, refer to [Configuration](/reference/edot-java/configuration.md). + +Configuration of those environment values depends on the deployment model. + +### Local EDOT Collector + +When deployed locally, the EDOT Collector is accessible with `http://localhost:4318` without authentication, no further configuration is required. The `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_HEADERS` environment variables do not have to be set. + +### Self-managed EDOT Collector + +When using a self-managed EDOT Collector, set the `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable to the OTLP endpoint of your self-managed EDOT Collector. If EDOT Collector requires authentication, set `OTEL_EXPORTER_OTLP_HEADERS` to include `Authorization=ApiKey `. + +### Elastic Managed OTLP endpoint + +Follow the [Serverless quickstart guides](docs-content://solutions/observability/get-started/opentelemetry/quickstart/serverless/index.md) to retrieve the `` and the ``. + +- Set `OTEL_EXPORTER_OTLP_ENDPOINT` to ``. +- Set `OTEL_EXPORTER_OTLP_HEADERS` to include `Authorization=ApiKey `. + +### Kubernetes + +Connection to the EDOT Collector is managed by the OTel Kubernetes Operator. [Follow the Quickstart Guides](docs-content://solutions/observability/get-started/opentelemetry/quickstart/index.md) for Kubernetes. + +## Run the Java agent + +Use the `-javaagent:` JVM argument with the path to agent jar. This requires to modify the JVM arguments and restart the application. + +```sh +java \ +-javaagent:/path/to/agent.jar \ +-jar myapp.jar +``` + +When modifying the JVM command line arguments is not possible, use the `JAVA_TOOL_OPTIONS` environment variable to provide the `-javaagent:` argument or JVM system properties. When `JAVA_TOOL_OPTIONS` is set, all JVMs automatically use it, so make sure to limit the scope to the relevant JVMs. + +Some application servers require manual steps or modification of their configuration files. Refer to [dedicated instructions](https://opentelemetry.io/docs/zero-code/java/agent/server-config/) for more details. + +For applications deployed with Kubernetes, use the [OpenTelemetry Operator](/reference/edot-java/setup/k8s.md). + +## Troubleshooting + +For help with common setup issues, refer to the [EDOT Java troubleshooting guide](docs-content://troubleshoot/ingest/opentelemetry/edot-sdks/java/index.md). \ No newline at end of file diff --git a/docs/reference/edot-java/setup/k8s.md b/docs/reference/edot-java/setup/k8s.md new file mode 100644 index 000000000..bf7560aa5 --- /dev/null +++ b/docs/reference/edot-java/setup/k8s.md @@ -0,0 +1,171 @@ +--- +navigation_title: Kubernetes Setup +description: Guide on instrumenting Java applications on Kubernetes using the OpenTelemetry Operator, EDOT Collectors, and the EDOT Java SDK. +applies_to: + stack: + serverless: + observability: + product: + edot_java: ga +products: + - id: cloud-serverless + - id: observability + - id: edot-sdk +--- + +# Instrumenting Java applications with EDOT SDKs on Kubernetes + +Learn how to instrument Java applications on Kubernetes using the OpenTelemetry Operator, the {{edot}} (EDOT) Collectors, and the EDOT Java SDK. + +- For general knowledge about the EDOT Java SDK, refer to the [EDOT Java Intro page](/reference/edot-java/index.md). +- For Java auto-instrumentation specifics, refer to [OpenTelemetry Operator Java auto-instrumentation](https://opentelemetry.io/docs/kubernetes/operator/automatic/#java). +- For general information about instrumenting applications on Kubernetes, refer to [instrumenting applications on Kubernetes](docs-content://solutions/observability/get-started/opentelemetry/use-cases/kubernetes/instrumenting-applications.md). + +## Extensions consideration + +The operator supports a configuration that installs [Java agent extensions](https://opentelemetry.io/docs/zero-code/java/agent/extensions/) in `Instrumentation` objects. The extension needs to be available in an image. Refer to [using extensions with the OpenTelemetry Java agent](https://www.elastic.co/observability-labs/blog/using-the-otel-operator-for-injecting-elastic-agents#using-an-extension-with-the-opentelemetry-java-agent) for an example of adding an extension to an agent. + +## Instrument a Java app + +Following this example, you can learn how to: + +- Turn on auto-instrumentation of a Java application using one of the following supported methods: + - Adding an annotation to the deployment Pods. + - Adding an annotation to the namespace. +- Verify that auto-instrumentation libraries are injected and configured correctly. +- Confirm data is flowing to {{kib}} Observability. + +In this example, the application you're instrumenting is a deployment named `java-app` running in the `java-ns` namespace. + +1. Ensure you have successfully [installed the OpenTelemetry Operator](docs-content://solutions/observability/get-started/opentelemetry/use-cases/kubernetes/deployment.md) and confirm that the following `Instrumentation` object exists in the system: + + ```sh + $ kubectl get instrumentation -n opentelemetry-operator-system + NAME AGE ENDPOINT + elastic-instrumentation 107s http://opentelemetry-kube-stack-daemon-collector.opentelemetry-operator-system.svc.cluster.local:4318 + ``` + + :::{note} + If your `Instrumentation` object has a different name or is created in a different namespace, you will have to adapt the annotation value in the next step. + ::: + +2. Turn on auto-instrumentation of your Java application using one of the following methods: + + - Edit your application workload definition and include the annotation under `spec.template.metadata.annotations`: + + ```yaml + kind: Deployment + metadata: + name: java-app + namespace: java-ns + spec: + # ... + template: + metadata: + # ... + annotations: + instrumentation.opentelemetry.io/inject-java: opentelemetry-operator-system/elastic-instrumentation + # ... + ``` + + - Alternatively, add the annotation at namespace level to apply auto-instrumentation in all Pods of the namespace: + + ```sh + kubectl annotate namespace java-ns instrumentation.opentelemetry.io/inject-java=opentelemetry-operator-system/elastic-instrumentation + ``` + +3. Restart the application + + After the annotation has been set, restart the application to create new Pods and inject the instrumentation libraries: + + ```sh + kubectl rollout restart deployment java-app -n java-ns + ``` + +4. Verify the [auto-instrumentation resources](docs-content://solutions/observability/get-started/opentelemetry/use-cases/kubernetes/instrumenting-applications.md#how-auto-instrumentation-works) are injected in the Pod: + + Run a `kubectl describe` of one of your application Pods and check: + + - There should be an init container named `opentelemetry-auto-instrumentation-java` in the Pod: + + ```sh + $ kubectl describe pod java-app-8d84c47b8-8h5z2 -n java-ns + Name: java-app-8d84c47b8-8h5z2 + Namespace: java-ns + ... + ... + Init Containers: + opentelemetry-auto-instrumentation-java: + Container ID: containerd://cbf67d7ca1bd62c25614b905a11e81405bed6fd215f2df21f84b90fd0279230b + Image: docker.elastic.co/observability/elastic-otel-javaagent:1.0.0 + Image ID: docker.elastic.co/observability/elastic-otel-javaagent@sha256:28d65d04a329c8d5545ed579d6c17f0d74800b7b1c5875e75e0efd29e210566a + Port: + Host Port: + Command: + cp + /javaagent.jar + /otel-auto-instrumentation-java/javaagent.jar + State: Terminated + Reason: Completed + Exit Code: 0 + Started: Wed, 13 Nov 2024 15:47:02 +0100 + Finished: Wed, 13 Nov 2024 15:47:03 +0100 + Ready: True + Restart Count: 0 + Environment: + Mounts: + /otel-auto-instrumentation-java from opentelemetry-auto-instrumentation-java (rw) + /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-swhn5 (ro) + ``` + + - The main container of the deployment is using the SDK as `javaagent`: + + ```bash + ... + Containers: + java-app: + Environment: + ... + JAVA_TOOL_OPTIONS: -javaagent:/otel-auto-instrumentation-java/javaagent.jar + OTEL_SERVICE_NAME: java-app + OTEL_EXPORTER_OTLP_ENDPOINT: http://opentelemetry-kube-stack-daemon-collector.opentelemetry-operator-system.svc.cluster.local:4318 + ... + ``` + + - The Pod has an `EmptyDir` volume named `opentelemetry-auto-instrumentation-java` mounted in both the main and the init containers in path `/otel-auto-instrumentation-java`. + + ```bash + Init Containers: + opentelemetry-auto-instrumentation-java: + ... + Mounts: + /otel-auto-instrumentation-java from opentelemetry-auto-instrumentation-java (rw) + Containers: + java-app: + ... + Mounts: + /otel-auto-instrumentation-java from opentelemetry-auto-instrumentation-java (rw) + ... + Volumes: + ... + opentelemetry-auto-instrumentation-java: + Type: EmptyDir (a temporary directory that shares a pod's lifetime) + ``` + + Ensure the environment variable `OTEL_EXPORTER_OTLP_ENDPOINT` points to a valid endpoint and there's network communication between the Pod and the endpoint. + +5. Confirm data is flowing to {{kib}}: + + - Open **Observability** → **Applications** → **Service inventory**, and determine if: + - The application appears in the list of services. + - The application shows transactions and metrics. + + - For application container logs, open {{kib}} Discover and filter for your Pods' logs. In the provided example, we could filter for them with either of the following: + - `k8s.deployment.name: "java-app"` (adapt the query filter to your use case) + - `k8s.pod.name: java-app*` (adapt the query filter to your use case) + + Note that the container logs are not provided by the instrumentation library, but by the DaemonSet collector deployed as part of the [operator installation](docs-content://solutions/observability/get-started/opentelemetry/use-cases/kubernetes/deployment.md). + +## Troubleshooting + +Refer to [troubleshoot auto-instrumentation](docs-content://solutions/observability/get-started/opentelemetry/use-cases/kubernetes/instrumenting-applications.md#troubleshooting-auto-instrumentation) for further analysis. diff --git a/docs/reference/edot-java/setup/runtime-attach.md b/docs/reference/edot-java/setup/runtime-attach.md new file mode 100644 index 000000000..646e0da01 --- /dev/null +++ b/docs/reference/edot-java/setup/runtime-attach.md @@ -0,0 +1,98 @@ +--- +navigation_title: Runtime attach Setup +description: Guide on instrumenting Java applications using EDOT Java runtime attach. +applies_to: + stack: + serverless: + observability: + product: + edot_java: ga +products: + - id: cloud-serverless + - id: observability + - id: edot-sdk +--- + +# Instrumenting Java applications using the EDOT Java runtime attach + +```{applies_to} +product: preview +``` + +Runtime attach includes the EDOT instrumentation agent in the application binary. This allows deploying the agent when access to JVM arguments or configuration is not possible, for example, with some managed services. The application development team can control the agent deployment and update cycle without having to modify the execution environment. Runtime attach only requires a minor modification of the application main entry point and one additional dependency. + +## Limitations + +The following limitations apply: + +- Runtime attach can't be used with multiple applications that share a JVM, for example, with web-applications and application servers. In this case, only the `-javaagent` setup option is supported. +- Agent update is tied to the application development and deployment lifecycle. You can't update application and agent independently. +- Agent can only be attached at application start; it can't be used to attach later during application runtime. +- Recent JVMs issue [warnings in standard error](#jvm-runtime-attach-warnings) and the feature might require explicit opt-in with JVM settings in the future. + +## Instrument a Java app + +Follow these steps to instrument your Java application using runtime attach. + +::::::{stepper} + +:::::{step} Add runtime attach to application dependencies + +::::{tab-set} + +:::{tab-item} Maven +```xml subs=true + + co.elastic.otel + elastic-otel-runtime-attach + {{version.edot_java}} + +``` +::: + +:::{tab-item} Gradle +```kotlin subs=true +implementation("co.elastic.otel:elastic-otel-runtime-attach:{{version.edot_java}}") +``` +::: + +:::: +::::: + +:::::{step} Modify the main method + +Add a single call to `RuntimeAttach.attachJavaagentToCurrentJvm()` at the start of the `main` method body. Here is an example of a simple spring-boot application: + +```java +@SpringBootApplication +public class MyApplication { + + public static void main(String[] args) { + RuntimeAttach.attachJavaagentToCurrentJvm(); + SpringApplication.run(MyApplication.class, args); + } +} +``` +::::: + +:::::{step} Package and redeploy the application + +After you've added the new dependency and modified the `main` method, package and redeploy your application. + +::::: +:::::: + +## JVM runtime attach warnings + +The following warning might appear in the process standard error output: + +``` +WARNING: A Java agent has been loaded dynamically (/tmp/otel-agent6227828786286549290/agent.jar) +WARNING: If a serviceability tool is in use, please run with -XX:+EnableDynamicAgentLoading to hide this warning +WARNING: If a serviceability tool is not in use, please run with -Djdk.instrument.traceUsage for more information +WARNING: Dynamic loading of agents will be disallowed by default in a future release +``` + +This message indicates that the dynamic agent attachment will be disabled by in the future. Add `-XX:+EnableDynamicAgentLoading` to the JVM arguments, or to the `JAVA_TOOL_OPTIONS` environment variable, to silence it. + +The runtime attach feature will be turned off by default in future JVM versions. Until then, you can ignore the warning. diff --git a/docs/reference/edot-java/supported-technologies.md b/docs/reference/edot-java/supported-technologies.md new file mode 100644 index 000000000..f8f99f342 --- /dev/null +++ b/docs/reference/edot-java/supported-technologies.md @@ -0,0 +1,57 @@ +--- +navigation_title: Supported Technologies +description: Overview of technologies supported by the Elastic Distribution of OpenTelemetry (EDOT) Java Agent, including JVM versions, application servers, frameworks, and LLM instrumentations. +applies_to: + stack: + serverless: + observability: + product: + edot_java: ga +products: + - id: cloud-serverless + - id: observability + - id: edot-sdk +--- + +# Technologies supported by the EDOT Java Agent + +The EDOT Java agent is a [distribution](https://opentelemetry.io/docs/concepts/distributions/) of [OpenTelemetry Java Instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation) agent. It inherits all the [supported](opentelemetry://reference/compatibility/nomenclature.md) technologies of the OpenTelemetry Java Instrumentation. + +## EDOT Collector and Elastic Stack versions + +The EDOT Java agent sends data through the OpenTelemetry protocol (OTLP). While OTLP ingest works with later 8.16+ versions of the EDOT Collector, for full support use either [EDOT Collector](elastic-agent://reference/edot-collector/index.md) versions 9.x or [{{serverless-full}}](docs-content://deploy-manage/deploy/elastic-cloud/serverless.md) for OTLP ingest. + +Refer to [EDOT SDKs compatibility](opentelemetry://reference/compatibility/sdks.md) for support details. + +:::{note} +Ingesting data from EDOT SDKs through EDOT Collector 9.x into Elastic Stack versions 8.18+ is supported. +::: + +## JVM versions + +The EDOT Java agent supports Java Virtual Machine (OpenJDK, OpenJ9) versions 8+. This follows from the [OpenTelemetry supported JVMs](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md#jvms-and-operating-systems). + +## JVM languages + +The EDOT Java agent is compatible with all JVM languages supported by the JVM version 8 and higher. + +## Application servers + +The EDOT Java agent supports [all the application servers documented by the OpenTelemetry Java agent](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md#application-servers). + +## Libraries and Frameworks instrumentations + +The EDOT Java agent supports [all the libraries and frameworks documented by the OpenTelemetry Java agent](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md#libraries--frameworks). + +Note that [some supported technologies are deactivated by default](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md#disabled-instrumentations) and need explicit configuration to be activated. + +The EDOT Java agent also supports technologies listed here that are not available in the [OpenTelemetry Java Instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation). + +Refer to the [EDOT Java agent configuration](/reference/edot-java/configuration.md#configuration-options) for defaults that might differ from the OpenTelemetry Java Instrumentation. + +## OpenAI Client instrumentation + +The minimum supported version of the OpenAI Java Client is 1.1.0. This instrumentation supports: + +* Tracing for requests, including GenAI-specific attributes such as token usage. +* Opt-in logging of OpenAI request and response content payloads. diff --git a/docs/reference/edot-java/toc.yml b/docs/reference/edot-java/toc.yml new file mode 100644 index 000000000..64bb7e1d3 --- /dev/null +++ b/docs/reference/edot-java/toc.yml @@ -0,0 +1,13 @@ +toc: + - file: index.md + - file: setup/index.md + children: + - file: setup/k8s.md + - file: setup/runtime-attach.md + - file: configuration.md + - file: features.md + - file: supported-technologies.md + - file: migration.md + - file: overhead.md + - title: Troubleshooting + crosslink: docs-content://troubleshoot/ingest/opentelemetry/edot-sdks/java/index.md \ No newline at end of file