|
1 | 1 | [[connect-to-elasticsearch]] |
2 | | -== Add data to {kib} |
| 2 | +== Add data |
| 3 | +++++ |
| 4 | +<titleabbrev>Add data</titleabbrev> |
| 5 | +++++ |
3 | 6 |
|
4 | | -To start working with your data in {kib}, you can: |
| 7 | +To start working with your data in {kib}, use one of the many ingest options, |
| 8 | +available from the home page. |
| 9 | +You can collect data from an app or service |
| 10 | +or upload a file that contains your data. If you're not ready to use your own data, |
| 11 | +add a sample data set and give {kib} a test drive. |
5 | 12 |
|
6 | | -* Upload a CSV, JSON, or log file with the File Data Visualizer. |
| 13 | +[role="screenshot"] |
| 14 | +image::images/add-data-home.png[Built-in options for adding data to Kibana: Add data, Add Elastic Agent, Upload a file] |
7 | 15 |
|
8 | | -* Upload geospatial data with the GeoJSON Upload feature. |
| 16 | +[float] |
| 17 | +[[add-data-tutorial-kibana]] |
| 18 | +=== Add data |
9 | 19 |
|
10 | | -* Index logs, metrics, events, or application data by setting up a Beats module. |
| 20 | +Want to ingest logs, metrics, security, or application data? |
| 21 | +Install and configure a Beats data shipper or other module to periodically collect the data |
| 22 | +and send it to {es}. You can then use the pre-built dashboards to explore and analyze the data. |
11 | 23 |
|
12 | | -* Connect {kib} with existing {es} indices. |
| 24 | +[role="screenshot"] |
| 25 | +image::images/add-data-tutorials.png[Add Data tutorials] |
13 | 26 |
|
14 | | -If you're not ready to use your own data, you can add a <<get-started, sample data set>> |
15 | | -to see all that you can do in {kib}. |
| 27 | +[discrete] |
| 28 | +=== Add Elastic Agent |
16 | 29 |
|
17 | | -[float] |
18 | | -[[upload-data-kibana]] |
19 | | -=== Upload a CSV, JSON, or log file |
20 | | - |
21 | | -experimental[] |
| 30 | +beta[] *Elastic Agent* is a sneak peek at the next generation of |
| 31 | +data integration modules, offering |
| 32 | +a centralized way to set up your integrations. |
| 33 | +With *Fleet*, you can add |
| 34 | +and manage integrations for popular services and platforms, providing |
| 35 | +an easy way to collect your data. The integrations |
| 36 | +ship with dashboards and visualizations, |
| 37 | +so you can quickly get insights into your data. |
22 | 38 |
|
23 | | -To visualize data in a CSV, JSON, or log file, you can upload it using the File |
24 | | -Data Visualizer. On the home page, click *Upload a file*, and |
25 | | -then drag your file onto the *File Data Visualizer*. Alternatively, you can open |
26 | | -it by navigating to *Machine Learning* from the side navigation and selecting |
27 | | -*Data Visualizer*. |
| 39 | +To get started, refer to |
| 40 | +{ingest-guide}/ingest-management-getting-started.html[Quick start: Get logs and metrics into the Elastic Stack]. |
28 | 41 |
|
29 | 42 | [role="screenshot"] |
30 | | -image::images/ingest-data.png[File Data Visualizer on the home page] |
| 43 | +image::images/add-data-fleet.png[Add data using Fleet] |
31 | 44 |
|
32 | | -You can upload a file up to 100 MB. This value is configurable up to 1 GB in |
33 | | -<<kibana-ml-settings, Advanced Settings>>. |
| 45 | +[discrete] |
| 46 | +[[upload-data-kibana]] |
| 47 | +=== Upload a file |
| 48 | + |
| 49 | +experimental[] If your data is in a CSV, JSON, or log file, you can upload it using the File |
| 50 | +Data Visualizer. You can upload a file up to 100 MB. This value is configurable up to 1 GB in |
| 51 | +<<kibana-ml-settings, Advanced Settings>>. To upload a file with geospatial data, |
| 52 | +refer to <<import-geospatial-data, Import geospatial data>>. |
34 | 53 |
|
35 | 54 | [role="screenshot"] |
36 | 55 | image::images/add-data-fv.png[File Data Visualizer] |
37 | 56 |
|
38 | | -The File Data Visualizer uses the {ref}/ml-find-file-structure.html[find_file_structure API] to analyze |
39 | | -the uploaded file and to suggest ingest pipelines and mappings for your data. |
| 57 | + |
40 | 58 |
|
41 | 59 | NOTE: This feature is not intended for use as part of a |
42 | 60 | repeated production process, but rather for the initial exploration of your data. |
43 | 61 |
|
44 | | -[float] |
45 | | -[[upload-geoipdata-kibana]] |
46 | | -=== Upload geospatial data |
47 | | - |
48 | | -To visualize geospatial data in a point or shape file, you can upload it using the <<import-geospatial-data, GeoJSON Upload>> |
49 | | -feature in Maps, and then use that data as a layer in a map. |
50 | | -The data is also available for use in the broader Kibana ecosystem, for example, |
51 | | -in visualizations and Canvas workpads. |
52 | | -With GeoJSON Upload, you can upload a file up to 50 MB. |
53 | | - |
54 | | -[float] |
55 | | -[[add-data-tutorial-kibana]] |
56 | | -=== Index metrics, log, security, and application data |
57 | 62 |
|
58 | | -The built-in data tutorials can help you quickly get up and running with |
59 | | -metrics data, log analytics, security events, and application data. |
60 | | -These tutorials walk you through installing and configuring a |
61 | | -Beats data shipper to periodically collect and send data to {es}. |
62 | | -You can then use the pre-built dashboards to explore and analyze the data. |
| 63 | +[discrete] |
| 64 | +=== Additional options for loading your data |
63 | 65 |
|
64 | | -You access the tutorials from the home page. |
65 | | -If a tutorial doesn’t exist for your data, go to the {beats-ref}/beats-reference.html[Beats overview] |
66 | | -to learn about other data shippers in the Beats family. |
| 66 | +If the {kib} ingest options don't work for you, you can index your |
| 67 | +data into Elasticsearch with {ref}/getting-started-index.html[REST APIs] |
| 68 | +or https://www.elastic.co/guide/en/elasticsearch/client/index.html[client libraries]. |
| 69 | +After you add your data, you're required to create an <<index-patterns,index pattern>> to tell |
| 70 | +{kib} where to find the data. |
67 | 71 |
|
68 | | -[role="screenshot"] |
69 | | -image::images/add-data-tutorials.png[Add Data tutorials] |
70 | | - |
71 | | - |
72 | | -[float] |
73 | | -[[connect-to-es]] |
74 | | -=== Connect with {es} indices |
75 | | - |
76 | | -To visualize data in existing {es} indices, you must |
77 | | -create an index pattern that matches the names of the indices that you want to explore. |
78 | | -When you add data with the File Data Visualizer, GeoJSON Upload feature, |
79 | | -or built-in tutorial, an index pattern is created for you. |
80 | | - |
81 | | -. Open the main menu, then click *Stack Management > Index Patterns*. |
82 | | - |
83 | | -. Click *Create index pattern*. |
84 | | - |
85 | | -. Specify an index pattern that matches the name of one or more of your Elasticsearch indices. |
86 | | -+ |
87 | | -For example, an index pattern can point to your Apache data from yesterday, |
88 | | -`filebeat-apache-4-3-2022`, or any index that matches the pattern, `filebeat-*`. |
89 | | -Using a wildcard is the more popular approach. |
90 | | - |
91 | | - |
92 | | -. Click *Next Step*, and then select the index field that contains the timestamp you want to use to perform time-based |
93 | | -comparisons. |
94 | | -+ |
95 | | -Kibana reads the index mapping and lists all fields that contain a timestamp. If your |
96 | | -index doesn't have time-based data, choose *I don't want to use the time filter*. |
97 | | -+ |
98 | | -You must select a time field to use global time filters on your dashboards. |
99 | | - |
100 | | -. Click *Create index pattern*. |
101 | | -+ |
102 | | -{kib} is now configured to access your {es} indices. |
103 | | -You’ll see a list of fields configured for the matching index. |
104 | | -You can designate your index pattern as the default by clicking the star icon on this page. |
105 | | -+ |
106 | | -When searching in *Discover* and creating visualizations, you choose a pattern |
107 | | -from the index pattern menu to specify the {es} indices that contain the data you want to explore. |
| 72 | +* To add data for Elastic Observability, refer to {observability-guide}/add-observability-data.html[Send data to Elasticsearch]. |
| 73 | +* To add data for Elastic Security, refer to https://www.elastic.co/guide/en/security/current/ingest-data.html[Ingest data to Elastic Security]. |
0 commit comments