Skip to content

[8.x](backport #41640) libbeat: increase total_fields.limit to 12500 #41657

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pierrehilbert merged 4 commits into from
Nov 19, 2024
Merged

[8.x](backport #41640) libbeat: increase total_fields.limit to 12500 #41657

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
52de213
libbeat: increase total_fields.limit to 12500 (#41640)
AndersonQ Nov 18, 2024
dc06c69
adjust changelog
AndersonQ Nov 18, 2024
5a2073a
fix changelog
AndersonQ Nov 18, 2024
2718179
Merge branch '8.x' into mergify/bp/8.x/pr-41640
AndersonQ Nov 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Filebeat now needs `dup3`, `faccessat2`, `prctl` and `setrlimit` syscalls to run the journald input. If this input is not being used, the syscalls are not needed. All Beats have those syscalls allowed now because the default seccomp policy is global to all Beats. {pull}40061[40061]
- Beats will rate limit the logs about errors when indexing events on Elasticsearch, logging a summary every 10s. The logs sent to the event log is unchanged. {issue}40157[40157]
- Drop support for Debian 10 and upgrade statically linked glibc from 2.28 to 2.31 {pull}41402[41402]
- Fix metrics not being ingested, due to "Limit of total fields [10000] has been exceeded while adding new fields [...]". The total fields limit has been increased to 12500. No significant performance impact on Elasticsearch is anticipated. {pull}41640[41640]

*Auditbeat*

Expand Down Expand Up @@ -371,7 +372,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Add metrics for the vSphere Virtualmachine metricset. {pull}40485[40485]
- Log the total time taken for GCP `ListTimeSeries` and `AggregatedList` requests {pull}40661[40661]
- Add metrics related to triggered alarms in all the vSphere metricsets. {pull}40714[40714] {pull}40876[40876]
- Add new metricset datastorecluster for vSphere module. {pull}40634[40634]
- Add new metricset datastorecluster for vSphere module. {pull}40634[40634]
- Add support for new metrics in datastorecluster metricset. {pull}40694[40694]
- Add metrics related to alert in all the vSphere metricsets. {pull}40714[40714]
- Add new metrics fot datastore and minor changes to overall vSphere metrics {pull}40766[40766]
Expand Down
2 changes: 1 addition & 1 deletion libbeat/template/load_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ func TestFileLoader_Load(t *testing.T) {
"refresh_interval": "5s",
"mapping": mapstr.M{
"total_fields": mapstr.M{
"limit": 10000,
"limit": defaultTotalFieldsLimit,
},
},
"query": mapstr.M{
Expand Down
2 changes: 1 addition & 1 deletion libbeat/template/template.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ import (
var (
// Defaults used in the template
defaultDateDetection = false
defaultTotalFieldsLimit = 10000
defaultTotalFieldsLimit = 12500
defaultMaxDocvalueFieldsSearch = 200

defaultFields []string
Expand Down