Skip to content

Jaeger configuration and deployment options #20643

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JStickler merged 9 commits into from
May 19, 2020
Merged

Jaeger configuration and deployment options #20643

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
e9fe881
[WIP] Jaeger configuration and deployment options
JStickler Mar 23, 2020
341c8fb
draft changes
JStickler Apr 6, 2020
ebd20ee
added sidecar injection topics
JStickler Apr 6, 2020
a974944
review comments
JStickler Apr 8, 2020
33bfe72
second round of review comments
JStickler Apr 8, 2020
bcbeaa7
further review comments
JStickler Apr 9, 2020
ba6ca7a
updating manual sidecar yaml
JStickler Apr 9, 2020
48d4b99
updating yaml again
JStickler Apr 9, 2020
859510e
review comments from Neal
JStickler May 14, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions _topic_map.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1466,14 +1466,16 @@ Topics:
- Name: Jaeger installation
Dir: jaeger_install
Topics:
- Name: Installing Jaeger
File: rhbjaeger-installation
# - Name: Deploying Jaeger
# File: rhbjaeger-deploying
- Name: Upgrading Jaeger
File: rhbjaeger-updating
- Name: Removing Jaeger
File: rhbjaeger-removing
# - Name: Jaeger architecture
# File: rhbjaeger-architecture
# - Name: Installing Jaeger
# File: rhbjaeger-installation
- Name: Deploying Jaeger
File: rhbjaeger-deploying
# - Name: Upgrading Jaeger
# File: rhbjaeger-updating
# - Name: Removing Jaeger
# File: rhbjaeger-removing

---
Name: Container-native virtualization
Expand Down
2 changes: 1 addition & 1 deletion installing/install_config/installation-types.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ Not all installation options are currently available for all platforms, as shown

|Custom
|xref:../../installing/installing_aws/installing-aws-user-infra.adoc#installing-aws-user-infra[X]
|xref:../../installing/installing_azure/installing-azure-user-infra.adoc#installing-azure-user-infra[X]
|
|xref:../../installing/installing_gcp/installing-gcp-user-infra.adoc#installing-gcp-user-infra[X]
|xref:../../installing/installing_openstack/installing-openstack-user.adoc#installing-openstack-user[X]
|
Expand Down
2 changes: 1 addition & 1 deletion installing/installing_azure/installing-azure-user-infra.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ in the Azure documentation.
include::modules/installation-azure-limits.adoc[leveloffset=+2]
include::modules/installation-azure-network-config.adoc[leveloffset=+2]

You can view Azure's DNS solution by visiting this
//You can view Azure's DNS solution by visiting this
xref:installation-azure-create-dns-zones_{context}[example for creating DNS zones].

include::modules/installation-azure-increasing-limits.adoc[leveloffset=+2]
Expand Down
55 changes: 43 additions & 12 deletions jaeger/jaeger_install/rhbjaeger-deploying.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,42 +1,73 @@
[id="deploying-jaeger"]
= Deploying Jaeger
include::modules/ossm-document-attributes.adoc[]
include::modules/jaeger-document-attributes.adoc[]
:context: jaeger-deploying

toc::[]

The Jaeger Operator includes a custom resource definition (CRD) file that determines the architecture and configuration settings to use when creating the Jaeger custom resources (CR). You can either install the default configuration or modify the custom resource file to better suit your business requirements.
The Jaeger Operator includes a custom resource definition (CRD) file that defines the architecture and configuration settings for the Jaeger resources. You can either install the default configuration or modify the file to better suit your business requirements.

Each Jaeger instance is associated with a deployment strategy. You define the strategy is defined in the custom resource file and when you create a Jaeger instance the Operator uses the deployment strategy and configuration options in the CR file to create the objects necessary for the deployment.
The Jaeger Operator currently supports the following deployment strategies:
Jaeger has pre-defined deployment strategies. You specify a deployment strategy in the custom resource file. When you create a Jaeger instance the Operator uses this configuration file to create the objects necessary for the deployment.

* *allInOne* (Default) strategy - This strategy is intended for development, testing, and demo purposes. The main backend components, Agent, Collector and Query service, are all packaged into a single executable which is configured (by default) to use in-memory storage.
.Jaeger custom resource file showing deployment strategy
[source,yaml]
----
apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
name: simple-prod
spec:
strategy: production <1>
----

<1> The Jaeger Operator currently supports the following deployment strategies:

* *allInOne* (Default) - This strategy is intended for development, testing, and demo purposes. The main backend components, Agent, Collector and Query service, are all packaged into a single executable which is configured (by default) to use in-memory storage.
+
[NOTE]
====
In-memory storage is not persistent, which means that if the Jaeger instance shuts down, restarts, or is replaced, that your trace data will be lost. For persistent storage, use the production Elasticsearch deployment strategy.
In-memory storage is not persistent, which means that if the Jaeger instance shuts down, restarts, or is replaced, that your trace data will be lost. And in-memory storage cannot be scaled, since each pod has its own memory. For persistent storage, you must use the `production` or `streaming` strategies, which use Elasticsearch as the default storage.
====

* *production* strategy - The production strategy is intended for production environments, where long term storage of trace data is important, as well as a more scalable and highly available architecture is required. Each of the backend components is therefore deployed separately. The Agent can be injected as a sidecar on the instrumented application or as a daemonset. The Query and Collector services are configured with a supported storage type - currently Elasticsearch. Multiple instances of each of these components can be provisioned as required for performance and resilience purposes. For more information about configuring Elasticsearch with {product-title}, see xref:../../logging/config/cluster-logging-elasticsearch.adoc[Configuring Elasticsearch].
* *production* - The production strategy is intended for production environments, where long term storage of trace data is important, as well as a more scalable and highly available architecture is required. Each of the backend components is therefore deployed separately. The Agent can be injected as a sidecar on the instrumented application or as a daemonset. The Query and Collector services are configured with a supported storage type - currently Elasticsearch. Multiple instances of each of these components can be provisioned as required for performance and resilience purposes.

* *streaming* strategy - The streaming strategy is designed to augment the production strategy by providing a streaming capability that effectively sits between the Collector and the backend storage (Elasticsearch). This provides the benefit of reducing the pressure on the backend storage, under high load situations, and enables other trace post-processing capabilities to tap into the real time span data directly from the streaming platform (Kafka).
* *streaming* - The streaming strategy is designed to augment the production strategy by providing a streaming capability that effectively sits between the Collector and the backend storage (Elasticsearch). This provides the benefit of reducing the pressure on the backend storage, under high load situations, and enables other trace post-processing capabilities to tap into the real time span data directly from the streaming platform (https://access.redhat.com/documentation/en-us/red_hat_amq/7.6/html/using_amq_streams_on_openshift/index[AMQ Streams]/ https://kafka.apache.org/documentation/[Kafka]).

[NOTE]
====
There are two ways to install Jaeger, as part of a service mesh or as a stand alone component. If you have installed Jaeger as part of Red Hat OpenShift Service Mesh, you must configure and deploy Jaeger as part of the xref:../../service_mesh/service_mesh_install/customizing-installation-ossm.adoc#customizing-installation-ossm[ServiceMeshControlPlane].
====



// The following include statements pull in the module files that comprise the assembly.

include::modules/jaeger-deploy-default.adoc[leveloffset=+1]

include::modules/jaeger-deploy-production-es.adoc[leveloffset=1]

include::modules/jaeger-deploy-streaming.adoc[leveloffset=1]

[id="customizing-jaeger-deployment"]
= Customizing Jaeger deployment

include::modules/jaeger-config-default.adoc[leveloffset=+1]

include::modules/jaeger-deploy-production-es.adoc[leveloffset=+1]
include::modules/jaeger-config-collector.adoc[leveloffset=+1]

include::modules/jaeger-config-sampling.adoc[leveloffset=+1]

include::modules/jaeger-config-storage.adoc[leveloffset=+1]

include::modules/jaeger-config-query.adoc[leveloffset=+1]

include::modules/jaeger-config-ingester.adoc[leveloffset=+1]

[id="injecting-sidecars"]
= Injecting sidecars

include::modules/jaeger-config-production-es.adoc[leveloffset=+1]
{ProductName} relies on a proxy sidecar within the application’s pod to provide the agent. The Jaeger Operator can inject Jaeger Agent sidecars into Deployment workloads. You can enable automatic sidecar injection or manage it manually.

include::modules/jaeger-deploy-streaming.adoc[leveloffset=+1]
include::modules/jaeger-sidecar-automatic.adoc[leveloffset=1]

include::modules/jaeger-config-streaming.adoc[leveloffset=+1]
include::modules/jaeger-sidecar-manual.adoc[leveloffset=1]
5 changes: 0 additions & 5 deletions jaeger/rhbjaeger-release-notes.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,6 @@ include::modules/jaeger-product-overview.adoc[leveloffset=+1]

include::modules/support.adoc[leveloffset=+1]

////
TODO
include::modules/jaeger-supported-configurations.adoc[leveloffset=+1]
////

include::modules/jaeger-rn-new-features.adoc[leveloffset=+1]

include::modules/jaeger-rn-known-issues.adoc[leveloffset=+1]
43 changes: 43 additions & 0 deletions modules/jaeger-config-collector.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
////
This REFERENCE module included in the following assemblies:
-rhbjaeger-deploy.adoc
////

[id="jaeger-config-collector_{context}"]
= Jaeger Collector configuration options

The Jaeger Collector is the component responsible for receiving the spans that were captured by the tracer and writing them to a persistent storage (Elasticsearch) when using the `production` strategy, or to AMQ Streams when using the `streaming` strategy.

The collectors are stateless and thus many instances of Jaeger Collector can be run in parallel. Collectors require almost no configuration, except for the location of the Elasticsearch cluster.
Copy link

@objectiser objectiser Mar 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we only support self provisioned ES clusters, the location of the ES cluster is not defined. However for the next version it will be supported (i.e. using external cluster) so need to take into account this will change for next release.

This is also an area where it is dependent upon the deployment strategy - i.e. if production then would be dealing with ES config, but if streaming then dealing with Kafka config.

Copy link

@objectiser objectiser Mar 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that the ES config options are described in a separate section. Possibly the Kafka options should also be described in a separate section (both showing the collector's kafka config and the ingester's kafka config) - with links from this section based on the strategy?

Then the options in this section only need to focus on the collector specific options, without worrying about the storage/kafka - which are strategy dependent.

In this case, there are additional collector options that need to be discussed: see https://www.jaegertracing.io/docs/1.17/cli/#jaeger-collector - this provides the options based on the storage chosen, but you should be able to pick any of them and just look for the options that are no related to that storage. Also don't include options that are marked as deprecated or experimental(?).

Copy link

@objectiser objectiser Mar 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be clear - the CLI flags section identifies the options which are defined in yaml in the options sections - but not all will be relevant. For example https://www.jaegertracing.io/docs/1.17/cli/#jaeger-collector-elasticsearch shows the CLI flags that are relevant when using the collector with elasticsearch storage. However the elasticsearch flags (es or es-archive prefixed) will need to be documented in the storage section, but others can be defined (with some exceptions) in the collector options node.

The general convention is usually to define the tree structure based on the '.', so --collector.grpc.tls.cert would be:

collector:
  options:
    collector:
      grpc:
        tls:
          cert: value

The exceptions that don't need to be documented are things like --help, --config-file, --sampling.strategies-file (as this is handled in the sampling config) and --span-storage.type (as handled in the storage section).


.Jaeger collector parameters
[options="header"]
[cols="l, a, a, a"]
|===
|Parameter |Description |Values |Default value
|spec:
collector:
options: {}
|Configuration options that define the Jaeger Collector.
|
|

|kafka:
producer:
topic: jaeger-spans
|The `topic` parameter identifies the Kafka configuration used by the collector to produce the messages, and the ingester to consume the messages.
|Label for the producer
|

|kafka:
producer:
brokers: my-cluster-kafka-brokers.kafka:9092
|Identifies the Kafka configuration used by the Collector to produce the messages. If brokers are not specified, and you have AMQ Streams 1.4.0+ installed, Jaeger will self-provision Kafka.
|
|

|log-level:
|Logging level for the collector.
|Possible values: `trace`, `debug`, `info`, `warning`, `error`, `fatal`, `panic`.
|
|===
126 changes: 126 additions & 0 deletions modules/jaeger-config-default.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
////
This REFERENCE module included in the following assemblies:
- rhbjaeger-deploying.adoc
////

[id="jaeger-config-default_{context}"]
= Jaeger default configuration options
:pantheon-module-type: REFERENCE

The Jaeger custom resource (CR) defines the architecture and settings to be used when creating the Jaeger resources. You can modify these parameters to customize your Jaeger implementation to your business needs.

.Jaeger generic YAML example
[source,yaml]
----
apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
name: name
spec:
strategy: <deployment_strategy>
allInOne:
options: {}
resources: {}
agent:
options: {}
resources: {}
collector:
options: {}
resources: {}
sampling:
options: {}
storage:
type:
options: {}
query:
options: {}
resources: {}
ingester:
options: {}
resources: {}
options: {}
----

.Jaeger parameters
[options="header"]
|===
|Parameter |Description |Values |Default value

|apiVersion:
|Version of the Application Program Interface to use when creating the object.
|`jaegertracing.io/v1`
|`jaegertracing.io/v1`

|kind:
|Defines the kind of Kubernetes object to create.
|`jaeger`
|

|metadata:
|Data that helps uniquely identify the object, including a `name` string, `UID`, and optional `namespace`.
|
|OpenShift automatically generates the `UID` and completes the `namespace` with the name of the project where the object is created.

|name:
|Name for the object.
|The name of your Jaeger instance.
|`jaeger-all-in-one-inmemory`

|spec:
|Specification for the object to be created.
|Contains all of the configuration parameters for your Jeager instance. When a common definition (for all Jaeger components) is required, it is defined under the spec node. When the definition relates to an individual component, it is placed under the spec/<component> node.
|N/A

|strategy:
|Jaeger deployment strategy
|`allInOne`, `production`, or `streaming`
|`allInOne`

|allInOne:
|Because the allInOne image deploys the agent, collector, query, ingester, Jaeger UI in a single pod, configuration for this deployment should nest component configuration under the allInOne parameter.
|
|

|agent:
|Configuration options that define the Jaeger agent.
|
|

|collector:
|Configuration options that define the Jaeger Collector.
|
|

|sampling:
|Configuration options that define the sampling strategies for tracing.
|
|

|storage:
|Configuration options that define the storage. All storage related options should be placed under `storage`, rather than under the `allInOne` or other component options.
|
|

|query:
|Configuration options that define the Query service.
|
|

|ingester:
|Configuration options that define the Ingester service.
|
|

|===


The following example YAML is the minimum required to create a Jaeger instance using the default settings.

.Example minimum required jaeger-all-in-one.yaml
[source,yaml]
----
apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
name: jaeger-all-in-one-inmemory
----
Loading