Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions docs/integrations-index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -134,8 +134,6 @@ include::{libbeat-dir}/shared-securing-beat.asciidoc[leveloffset=+1]

include::{libbeat-dir}/monitoring/monitoring-beats.asciidoc[leveloffset=+1]

include::./legacy/exploring-es-data.asciidoc[leveloffset=+1]

include::./legacy/troubleshooting.asciidoc[leveloffset=+1]

// include::./legacy/breaking-changes.asciidoc[leveloffset=+1]
Expand Down
13 changes: 0 additions & 13 deletions docs/legacy/error-indices.asciidoc

This file was deleted.

111 changes: 60 additions & 51 deletions docs/legacy/exploring-es-data.asciidoc
Original file line number Diff line number Diff line change
@@ -1,58 +1,16 @@
[[exploring-es-data]]
= Explore data in {es}

Elastic APM data is stored in data streams.
include::../data-streams.asciidoc[tag=data-streams]

The namespace default is `default`.
To configure a custom namespace, set `data_streams.namespace`:

[source,yaml]
----
apm-server:
data_streams.namespace: custom_namespace
----

[discrete]
[[apm-data-streams-list-standalone]]
== APM data streams

By type, the APM data streams are:

Traces::
Traces are comprised of {apm-guide-ref}/data-model.html[spans and transactions].
Traces are stored in the following data streams:
+
include::../data-streams.asciidoc[tag=traces-data-streams]

Metrics::
Metrics include application-based metrics, aggregation metrics, and basic system metrics.
Metrics are stored in the following data streams:
+
include::../data-streams.asciidoc[tag=metrics-data-streams]

Logs::
Logs include application error events and application logs.
Logs are stored in the following data streams:
+
include::../data-streams.asciidoc[tag=logs-data-streams]

[float]
[[sample-apm-document]]
== Sample APM documents

Sample documents for each of the APM event types are available on these pages:

* <<transaction-indices>>
* <<span-indices>>
* <<error-indices>>
* <<elasticsearch-query-examples>>
* <<sample-apm-document>>
* <<metricset-indices>>
* <<sourcemap-indices>>

[float]
[[elasticsearch-query-examples]]
== {es} query examples

Elastic APM data is stored in <<apm-data-streams>>.

The following examples enable you to interact with {es}'s REST API.
One possible way to do this is using {kib}'s
{kibana-ref}/console-kibana.html[{dev-tools-app} console].
Expand Down Expand Up @@ -90,9 +48,60 @@ GET /_template/your-template-name
----
// CONSOLE

[float]
[[sample-apm-document]]
== Sample APM documents

Sample documents for each of the APM event types are available below:

[%collapsible]
.Transaction documents
====
Example transaction documents indexed in {es}:

[source,json]
----
include::../data/elasticsearch/generated/transactions.json[]
----
====

[%collapsible]
.Span documents
====
Example span documents indexed in {es}:

[source,json]
----
include::../data/elasticsearch/generated/spans.json[]
----
====

[%collapsible]
.Error documents
====
Example error documents indexed in {es}:

[source,json]
----
include::../data/elasticsearch/generated/errors.json[]
----
====

[%collapsible]
.Metric document
====
include::./metricset-indices.asciidoc[tag=example]
====

[%collapsible]
.Source map documents
====
Example source map document indexed in {es}:

[source,json]
----
include::../data/intake-api/generated/sourcemap/bundle.js.map[]
----
====

include::./transaction-indices.asciidoc[]
include::./span-indices.asciidoc[]
include::./error-indices.asciidoc[]
include::./metricset-indices.asciidoc[]
include::./sourcemap-indices.asciidoc[]
include::./metricset-indices.asciidoc[]
8 changes: 3 additions & 5 deletions docs/legacy/metricset-indices.asciidoc
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
[[metricset-indices]]
== Metrics documents

++++
<titleabbrev>Metrics documents</titleabbrev>
++++
== Application metrics
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO we'll need to rework the metrics docs for some reasons:

  • the main question that we see raised is which metrics end up in service specific datastreams and which in the generic one
  • irritation around the aggregated metrics

We'll need to better cover these topics, but without over-documenting it, as the concrete (aggregated) metrics data streams are considered an internal detail, that we might change, as long as the UI changes the queries in a compatible way.
Should we have an in-depth conversation for this topic on this PR, or do you prefer a follow up issue to not slow down the structural changes?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Makes sense. I'd prefer to move the conversation to a separate issue.


APM Server stores application metrics sent by agents as documents in {es}.
Metric documents contain a timestamp, one or more metric fields,
Expand Down Expand Up @@ -129,6 +125,7 @@ The `@timestamp` field of these documents holds the start of the aggregation int
[[example-metric-document]]
=== Example metric document

// tag::example[]
Below is an example of a metric document as stored in {es}, containing JVM metrics produced by the {apm-java-agent}.
The document contains two related metrics: `jvm.gc.time` and `jvm.gc.count`. These are accompanied by various fields describing
the environment in which the metrics were captured: service name, host name, Kubernetes pod UID, container ID, process ID, and more.
Expand All @@ -138,3 +135,4 @@ These fields make it possible to search and aggregate across various dimensions,
----
include::../data/elasticsearch/metricset.json[]
----
// end::example[]
13 changes: 0 additions & 13 deletions docs/legacy/sourcemap-indices.asciidoc

This file was deleted.

13 changes: 0 additions & 13 deletions docs/legacy/span-indices.asciidoc

This file was deleted.

13 changes: 0 additions & 13 deletions docs/legacy/transaction-indices.asciidoc

This file was deleted.

2 changes: 2 additions & 0 deletions docs/manage-storage.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -206,3 +206,5 @@ POST /.ds-*-apm*/_update_by_query?expand_wildcards=all
// CONSOLE

TIP: Remember to also change the service name in the {apm-agents-ref}/index.html[{apm-agent} configuration].

include::./legacy/exploring-es-data.asciidoc[leveloffset=+2]