Skip to content
This repository was archived by the owner on Apr 28, 2025. It is now read-only.

Refactor limits and overrides #200

Merged
merged 1 commit into from
Oct 12, 2020
Merged

Refactor limits and overrides #200

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## master / unreleased

* [CHANGE] Add the default preset 'extra_small_user' and reference it in the CLI flags. This will raise the limits of the 'small_user' preset to the defaults for `ingester.max-samples-per-query` and `ingester.max-series-per-query`. #200
* [FEATURE] Add overrides-exporter as optional deployment to expose configured runtime overrides and presets. #198

## 1.4.0 / 2020-10-02
Expand Down
53 changes: 27 additions & 26 deletions cortex/config.libsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -274,39 +274,40 @@
fallback_config: {},
},

// === Per-tenant usage limits. ===
//
// These are the defaults. Distributor limits will be 5x (#replicas) higher,
// ingester limits are 6s (#replicas) / 3x (#replication factor) higher.
limits: $._config.overrides.extra_small_user,

overrides: {
// === Per-tenant usage limits. ===
//
// These are the defaults. Distributor limits will be 5x (#replicas) higher,
// ingester limits are 6s (#replicas) / 3x (#replication factor) higher.
//
// extra_small_user: {
// ingestion_rate: 10,000
// ingestion_burst_size: 200,000
//
// max_series_per_user: 0 (disabled)
// max_series_per_metric: 0 (disabled)
//
// // Our limit should be 100k, but we need some room of about ~50% to take rollouts into account
// max_global_series_per_user: 150,000
// max_global_series_per_metric: 20,000
//
// max_series_per_query: 10,000
// max_samples_per_query: 100,000
// },
extra_small_user:: {
max_series_per_user: 0, // Disabled in favour of the max global limit
max_series_per_metric: 0, // Disabled in favour of the max global limit

small_user:: {
ingestion_rate: 100000,
ingestion_burst_size: 1000000,
// Our limit should be 100k, but we need some room of about ~50% to take rollouts into account
max_global_series_per_user: 150000,
max_global_series_per_metric: 20000,

max_series_per_user: 0,
max_series_per_metric: 0,
max_series_per_query: 100000,
max_samples_per_query: 1000000,

ingestion_rate: 10000,
ingestion_burst_size: 200000,
},

small_user:: {
max_series_per_metric: 0, // Disabled in favour of the max global limit
max_series_per_user: 0, // Disabled in favour of the max global limit

max_global_series_per_user: 1000000,
max_global_series_per_metric: 100000,

max_series_per_query: 10000,
max_samples_per_query: 100000,
max_series_per_query: 100000,
max_samples_per_query: 1000000,

ingestion_rate: 100000,
ingestion_burst_size: 1000000,
},

medium_user:: {
Expand Down
4 changes: 2 additions & 2 deletions cortex/distributor.libsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@
'server.grpc.keepalive.max-connection-idle': '1m',

'distributor.ingestion-rate-limit-strategy': 'global',
'distributor.ingestion-rate-limit': 10000, // 10K
'distributor.ingestion-burst-size': 200000, // 200k
'distributor.ingestion-rate-limit': $._config.limits.ingestion_rate,
'distributor.ingestion-burst-size': $._config.limits.ingestion_burst_size,

// The ingestion rate global limit requires the distributors to form a ring.
'distributor.ring.consul.hostname': 'consul.%s.svc.cluster.local:8500' % $._config.namespace,
Expand Down
10 changes: 6 additions & 4 deletions cortex/ingester.libsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,12 @@

// Limits config.
'ingester.max-chunk-idle': $._config.max_chunk_idle,
'ingester.max-global-series-per-user': 150000, // 150K
'ingester.max-global-series-per-metric': 20000, // 20K
'ingester.max-series-per-user': 0, // Disabled in favour of the max global limit
'ingester.max-series-per-metric': 0, // Disabled in favour of the max global limit
'ingester.max-series-per-user': $._config.limits.max_series_per_user,
'ingester.max-series-per-metric': $._config.limits.max_series_per_metric,
'ingester.max-global-series-per-user': $._config.limits.max_global_series_per_user,
'ingester.max-global-series-per-metric': $._config.limits.max_global_series_per_metric,
'ingester.max-series-per-query': $._config.limits.max_series_per_query,
'ingester.max-samples-per-query': $._config.limits.max_samples_per_query,
'limits.per-user-override-config': '/etc/cortex/overrides.yaml',
'server.grpc-max-concurrent-streams': 100000,
} + (
Expand Down
3 changes: 3 additions & 0 deletions cortex/overrides-exporter.libsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,10 @@
_config+: {
// overrides exporter can also make the configured presets available, this
// list references entries within $._config.overrides

overrides_exporter_presets:: [
'extra_small_user',
'small_user',
'medium_user',
'big_user',
'super_user',
Expand Down