Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions docs/integrations-index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -137,8 +137,6 @@ include::{libbeat-dir}/shared-securing-beat.asciidoc[leveloffset=+1]

include::{libbeat-dir}/monitoring/monitoring-beats.asciidoc[leveloffset=+1]

include::./legacy/exploring-es-data.asciidoc[leveloffset=+1]

include::./legacy/troubleshooting.asciidoc[leveloffset=+1]

// include::./legacy/breaking-changes.asciidoc[leveloffset=+1]
Expand Down
13 changes: 0 additions & 13 deletions docs/legacy/error-indices.asciidoc

This file was deleted.

111 changes: 60 additions & 51 deletions docs/legacy/exploring-es-data.asciidoc
Original file line number Diff line number Diff line change
@@ -1,58 +1,16 @@
[[exploring-es-data]]
= Explore data in {es}

Elastic APM data is stored in data streams.
include::../data-streams.asciidoc[tag=data-streams]

The namespace default is `default`.
To configure a custom namespace, set `data_streams.namespace`:

[source,yaml]
----
apm-server:
data_streams.namespace: custom_namespace
----

[discrete]
[[apm-data-streams-list-standalone]]
== APM data streams

By type, the APM data streams are:

Traces::
Traces are comprised of {apm-guide-ref}/data-model.html[spans and transactions].
Traces are stored in the following data streams:
+
include::../data-streams.asciidoc[tag=traces-data-streams]

Metrics::
Metrics include application-based metrics, aggregation metrics, and basic system metrics.
Metrics are stored in the following data streams:
+
include::../data-streams.asciidoc[tag=metrics-data-streams]

Logs::
Logs include application error events and application logs.
Logs are stored in the following data streams:
+
include::../data-streams.asciidoc[tag=logs-data-streams]

[float]
[[sample-apm-document]]
== Sample APM documents

Sample documents for each of the APM event types are available on these pages:

* <<transaction-indices>>
* <<span-indices>>
* <<error-indices>>
* <<elasticsearch-query-examples>>
* <<sample-apm-document>>
* <<metricset-indices>>
* <<sourcemap-indices>>

[float]
[[elasticsearch-query-examples]]
== {es} query examples

Elastic APM data is stored in <<apm-data-streams>>.

The following examples enable you to interact with {es}'s REST API.
One possible way to do this is using {kib}'s
{kibana-ref}/console-kibana.html[{dev-tools-app} console].
Expand Down Expand Up @@ -90,9 +48,60 @@ GET /_template/your-template-name
----
// CONSOLE

[float]
[[sample-apm-document]]
== Sample APM documents

Sample documents for each of the APM event types are available below:

[%collapsible]
.Transaction documents
====
Example transaction documents indexed in {es}:

[source,json]
----
include::../data/elasticsearch/generated/transactions.json[]
----
====

[%collapsible]
.Span documents
====
Example span documents indexed in {es}:

[source,json]
----
include::../data/elasticsearch/generated/spans.json[]
----
====

[%collapsible]
.Error documents
====
Example error documents indexed in {es}:

[source,json]
----
include::../data/elasticsearch/generated/errors.json[]
----
====

[%collapsible]
.Metric document
====
include::./metricset-indices.asciidoc[tag=example]
====

[%collapsible]
.Source map documents
====
Example source map document indexed in {es}:

[source,json]
----
include::../data/intake-api/generated/sourcemap/bundle.js.map[]
----
====

include::./transaction-indices.asciidoc[]
include::./span-indices.asciidoc[]
include::./error-indices.asciidoc[]
include::./metricset-indices.asciidoc[]
include::./sourcemap-indices.asciidoc[]
include::./metricset-indices.asciidoc[]
8 changes: 3 additions & 5 deletions docs/legacy/metricset-indices.asciidoc
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
[[metricset-indices]]
== Metrics documents

++++
<titleabbrev>Metrics documents</titleabbrev>
++++
== Application metrics

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO we'll need to rework the metrics docs for some reasons:

  • the main question that we see raised is which metrics end up in service specific datastreams and which in the generic one
  • irritation around the aggregated metrics

We'll need to better cover these topics, but without over-documenting it, as the concrete (aggregated) metrics data streams are considered an internal detail, that we might change, as long as the UI changes the queries in a compatible way.
Should we have an in-depth conversation for this topic on this PR, or do you prefer a follow up issue to not slow down the structural changes?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Makes sense. I'd prefer to move the conversation to a separate issue.


APM Server stores application metrics sent by agents as documents in {es}.
Metric documents contain a timestamp, one or more metric fields,
Expand Down Expand Up @@ -129,6 +125,7 @@ The `@timestamp` field of these documents holds the start of the aggregation int
[[example-metric-document]]
=== Example metric document

// tag::example[]
Below is an example of a metric document as stored in {es}, containing JVM metrics produced by the {apm-java-agent}.
The document contains two related metrics: `jvm.gc.time` and `jvm.gc.count`. These are accompanied by various fields describing
the environment in which the metrics were captured: service name, host name, Kubernetes pod UID, container ID, process ID, and more.
Expand All @@ -138,3 +135,4 @@ These fields make it possible to search and aggregate across various dimensions,
----
include::../data/elasticsearch/metricset.json[]
----
// end::example[]
13 changes: 0 additions & 13 deletions docs/legacy/sourcemap-indices.asciidoc

This file was deleted.

13 changes: 0 additions & 13 deletions docs/legacy/span-indices.asciidoc

This file was deleted.

13 changes: 0 additions & 13 deletions docs/legacy/transaction-indices.asciidoc

This file was deleted.

2 changes: 2 additions & 0 deletions docs/manage-storage.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -208,3 +208,5 @@ POST /.ds-*-apm*/_update_by_query?expand_wildcards=all
TIP: Remember to also change the service name in the {apm-agents-ref}/index.html[{apm-agent} configuration].

include::./apm-tune-elasticsearch.asciidoc[]

include::./legacy/exploring-es-data.asciidoc[leveloffset=+2]