Skip to content

Commit a55c869

Browse files
ywelschDaveCTurnerjrodewig
committed
Properly document keepalive and other tcp options (#60216)
Keepalive options are not well-documented (only in transport section, although also available at http and network level). Co-authored-by: David Turner <[email protected]> Co-authored-by: James Rodewig <[email protected]>
1 parent ffe114b commit a55c869

File tree

7 files changed

+149
-48
lines changed

7 files changed

+149
-48
lines changed

‎docs/reference/migration/migrate_7_10.asciidoc‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,4 +29,15 @@ checked this limit when `docvalue_fields` appeared in the top-level search
2929
request body. Now we also enforce the limit when loading doc value fields in
3030
an `inner_hits` section or in a `top_hits` aggregation.
3131

32+
[discrete]
33+
[[breaking_710_networking_changes]]
34+
=== Networking Changes
35+
36+
[discrete]
37+
==== *.tcp.keep_idle and *.tcp.keep_interval now bounded
38+
The settings `{network,transport,http}.tcp.keep_idle` and
39+
`{network,transport,http}.tcp.keep_interval` are now limited to a maximum
40+
value of 300, which amounts to 5 minutes. This helps ensure that, if
41+
explicitly configured, only reasonable values are set.
42+
3243
//end::notable-breaking-changes[]

‎docs/reference/modules/http.asciidoc‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,44 @@ The maximum number of warning headers in client HTTP responses. Defaults to `unb
127127
`http.max_warning_header_size`::
128128
The maximum total size of warning headers in client HTTP responses. Defaults to `unbounded`.
129129

130+
`http.tcp.no_delay`::
131+
Enable or disable the https://en.wikipedia.org/wiki/Nagle%27s_algorithm[TCP no delay]
132+
setting. Defaults to `network.tcp.no_delay`.
133+
134+
`http.tcp.keep_alive`::
135+
Configures the `SO_KEEPALIVE` option for this socket, which
136+
determines whether it sends TCP keepalive probes.
137+
Defaults to `network.tcp.keep_alive`.
138+
139+
`http.tcp.keep_idle`:: Configures the `TCP_KEEPIDLE` option for this socket, which
140+
determines the time in seconds that a connection must be idle before
141+
starting to send TCP keepalive probes. Defaults to `network.tcp.keep_idle`, which
142+
uses the system default. This value cannot exceed `300` seconds. Only applicable on
143+
Linux and macOS, and requires Java 11 or newer.
144+
145+
`http.tcp.keep_interval`:: Configures the `TCP_KEEPINTVL` option for this socket,
146+
which determines the time in seconds between sending TCP keepalive probes.
147+
Defaults to `network.tcp.keep_interval`, which uses the system default.
148+
This value cannot exceed `300` seconds. Only applicable on Linux and macOS, and requires
149+
Java 11 or newer.
150+
151+
`http.tcp.keep_count`:: Configures the `TCP_KEEPCNT` option for this socket, which
152+
determines the number of unacknowledged TCP keepalive probes that may be
153+
sent on a connection before it is dropped. Defaults to `network.tcp.keep_count`,
154+
which uses the system default. Only applicable on Linux and macOS, and
155+
requires Java 11 or newer.
156+
157+
`http.tcp.reuse_address`::
158+
Should an address be reused or not. Defaults to `network.tcp.reuse_address`.
159+
160+
`http.tcp.send_buffer_size`::
161+
The size of the TCP send buffer (specified with <<size-units,size units>>).
162+
Defaults to `network.tcp.send_buffer_size`.
163+
164+
`http.tcp.receive_buffer_size`::
165+
The size of the TCP receive buffer (specified with <<size-units,size units>>).
166+
Defaults to `network.tcp.receive_buffer_size`.
167+
130168
[http-rest-request-tracer]
131169
==== REST request tracer
132170

‎docs/reference/modules/network.asciidoc‎

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -130,8 +130,25 @@ Enable or disable the https://en.wikipedia.org/wiki/Nagle%27s_algorithm[TCP no d
130130
setting. Defaults to `true`.
131131

132132
`network.tcp.keep_alive`::
133-
Enable or disable https://en.wikipedia.org/wiki/Keepalive[TCP keep alive].
134-
Defaults to `true`.
133+
Configures the `SO_KEEPALIVE` option for this socket, which
134+
determines whether it sends TCP keepalive probes.
135+
136+
`network.tcp.keep_idle`:: Configures the `TCP_KEEPIDLE` option for this socket, which
137+
determines the time in seconds that a connection must be idle before
138+
starting to send TCP keepalive probes. Defaults to `-1`, which uses
139+
the system default. This value cannot exceed `300` seconds. Only applicable on Linux and macOS,
140+
and requires Java 11 or newer.
141+
142+
`network.tcp.keep_interval`:: Configures the `TCP_KEEPINTVL` option for this socket,
143+
which determines the time in seconds between sending TCP keepalive probes.
144+
Defaults to `-1`, which uses the system default. This value cannot exceed `300` seconds.
145+
Only applicable on Linux and macOS, and requires Java 11 or newer.
146+
147+
`network.tcp.keep_count`:: Configures the `TCP_KEEPCNT` option for this socket, which
148+
determines the number of unacknowledged TCP keepalive probes that may be
149+
sent on a connection before it is dropped. Defaults to `-1`,
150+
which uses the system default. Only applicable on Linux and macOS, and requires
151+
Java 11 or newer.
135152

136153
`network.tcp.reuse_address`::
137154
Should an address be reused or not. Defaults to `true` on non-windows

‎docs/reference/modules/transport.asciidoc‎

Lines changed: 69 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -18,44 +18,90 @@ as search in Elasticsearch.
1818
[[transport-settings]]
1919
==== Transport settings
2020

21-
The internal transport communicates over TCP. You can configure it with the
22-
following settings:
21+
The following settings can be configured for the internal transport that
22+
communicates over TCP. These settings also use the common
23+
<<modules-network,network settings>>.
2324

24-
[cols="<,<",options="header",]
25-
|=======================================================================
26-
|Setting |Description
27-
|`transport.port` |A bind port range. Defaults to `9300-9400`.
25+
`transport.port`::
26+
A bind port range. Defaults to `9300-9400`.
2827

29-
|`transport.publish_port` |The port that other nodes in the cluster
28+
`transport.publish_port`::
29+
The port that other nodes in the cluster
3030
should use when communicating with this node. Useful when a cluster node
3131
is behind a proxy or firewall and the `transport.port` is not directly
3232
addressable from the outside. Defaults to the actual port assigned via
3333
`transport.port`.
3434

35-
|`transport.bind_host` |The host address to bind the transport service to. Defaults to `transport.host` (if set) or `network.bind_host`.
36-
37-
|`transport.publish_host` |The host address to publish for nodes in the cluster to connect to. Defaults to `transport.host` (if set) or `network.publish_host`.
35+
`transport.bind_host`::
36+
The host address to bind the transport service to. Defaults to
37+
`transport.host` (if set) or `network.bind_host`.
3838

39-
|`transport.host` |Used to set the `transport.bind_host` and the `transport.publish_host`.
39+
`transport.publish_host`::
40+
The host address to publish for nodes in the cluster to connect to.
41+
Defaults to `transport.host` (if set) or `network.publish_host`.
4042

43+
`transport.host`::
44+
Used to set the `transport.bind_host` and the `transport.publish_host`.
4145

42-
|`transport.connect_timeout` |The connect timeout for initiating a new connection (in
46+
`transport.connect_timeout`::
47+
The connect timeout for initiating a new connection (in
4348
time setting format). Defaults to `30s`.
4449

45-
|`transport.compress` |Set to `true` to enable compression (`DEFLATE`) between
50+
`transport.compress`::
51+
Set to `true` to enable compression (`DEFLATE`) between
4652
all nodes. Defaults to `false`.
4753

48-
|`transport.ping_schedule` | Schedule a regular application-level ping message
54+
`transport.ping_schedule`::
55+
Schedule a regular application-level ping message
4956
to ensure that transport connections between nodes are kept alive. Defaults to
5057
`5s` in the transport client and `-1` (disabled) elsewhere. It is preferable
5158
to correctly configure TCP keep-alives instead of using this feature, because
5259
TCP keep-alives apply to all kinds of long-lived connections and not just to
5360
transport connections.
5461

55-
|=======================================================================
56-
57-
It also uses the common
58-
<<modules-network,network settings>>.
62+
`transport.tcp.no_delay`::
63+
Enable or disable the https://en.wikipedia.org/wiki/Nagle%27s_algorithm[TCP no delay]
64+
setting. Defaults to `network.tcp.no_delay`.
65+
66+
`transport.tcp.keep_alive`::
67+
Configures the `SO_KEEPALIVE` option for this socket, which
68+
determines whether it sends TCP keepalive probes.
69+
Defaults to `network.tcp.keep_alive`.
70+
71+
`transport.tcp.keep_idle`::
72+
Configures the `TCP_KEEPIDLE` option for this socket, which
73+
determines the time in seconds that a connection must be idle before
74+
starting to send TCP keepalive probes. Defaults to `network.tcp.keep_idle` if set,
75+
or the system default otherwise.
76+
This value cannot exceed `300` seconds. In cases where the system default
77+
is higher than `300`, the value is automatically lowered to `300`. Only applicable on
78+
Linux and macOS, and requires Java 11 or newer.
79+
80+
`transport.tcp.keep_interval`::
81+
Configures the `TCP_KEEPINTVL` option for this socket,
82+
which determines the time in seconds between sending TCP keepalive probes.
83+
Defaults to `network.tcp.keep_interval` if set, or the system default otherwise.
84+
This value cannot exceed `300` seconds. In cases where the system default is higher than `300`,
85+
the value is automatically lowered to `300`. Only applicable on Linux and macOS,
86+
and requires Java 11 or newer.
87+
88+
`transport.tcp.keep_count`::
89+
Configures the `TCP_KEEPCNT` option for this socket, which
90+
determines the number of unacknowledged TCP keepalive probes that may be
91+
sent on a connection before it is dropped. Defaults to `network.tcp.keep_count`
92+
if set, or the system default otherwise. Only applicable on Linux and macOS, and
93+
requires Java 11 or newer.
94+
95+
transport.tcp.reuse_address::
96+
Should an address be reused or not. Defaults to `network.tcp.reuse_address`.
97+
98+
`transport.tcp.send_buffer_size`::
99+
The size of the TCP send buffer (specified with <<size-units,size units>>).
100+
Defaults to `network.tcp.send_buffer_size`.
101+
102+
`transport.tcp.receive_buffer_size`::
103+
The size of the TCP receive buffer (specified with <<size-units,size units>>).
104+
Defaults to `network.tcp.receive_buffer_size`.
59105

60106
[[transport-profiles]]
61107
===== Transport profiles
@@ -83,27 +129,11 @@ example above:
83129
* `port`: The port to which to bind.
84130
* `bind_host`: The host to which to bind.
85131
* `publish_host`: The host which is published in informational APIs.
86-
* `tcp.no_delay`: Configures the `TCP_NO_DELAY` option for this socket.
87-
* `tcp.keep_alive`: Configures the `SO_KEEPALIVE` option for this socket, which
88-
determines whether it sends TCP keepalive probes.
89-
* `tcp.keep_idle`: Configures the `TCP_KEEPIDLE` option for this socket, which
90-
determines the time in seconds that a connection must be idle before
91-
starting to send TCP keepalive probes. Defaults to `-1` which means to use
92-
the smaller of 300 or the system default. May not be greater than 300. Only
93-
applicable on Linux and macOS, and requires Java 11 or newer.
94-
* `tcp.keep_interval`: Configures the `TCP_KEEPINTVL` option for this socket,
95-
which determines the time in seconds between sending TCP keepalive probes.
96-
Defaults to `-1` which means to use the smaller of 300 or the system
97-
default. May not be greater than 300. Only applicable on Linux and macOS,
98-
and requires Java 11 or newer.
99-
* `tcp.keep_count`: Configures the `TCP_KEEPCNT` option for this socket, which
100-
determines the number of unacknowledged TCP keepalive probes that may be
101-
sent on a connection before it is dropped. Defaults to `-1` which means to
102-
use the system default. Only applicable on Linux and macOS, and requires
103-
Java 11 or newer.
104-
* `tcp.reuse_address`: Configures the `SO_REUSEADDR` option for this socket.
105-
* `tcp.send_buffer_size`: Configures the send buffer size of the socket.
106-
* `tcp.receive_buffer_size`: Configures the receive buffer size of the socket.
132+
133+
Profiles also support all the other transport settings specified in the
134+
<<transport-settings,transport settings>> section, and use these as defaults.
135+
For example, `transport.profiles.client.tcp.reuse_address` can be explicitly
136+
configured, and defaults otherwise to `transport.tcp.reuse_address`.
107137

108138
[[long-lived-connections]]
109139
===== Long-lived idle connections

‎server/src/main/java/org/elasticsearch/common/settings/Setting.java‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1182,6 +1182,11 @@ public static Setting<Integer> intSetting(String key, Setting<Integer> fallbackS
11821182
return new Setting<>(key, fallbackSetting, (s) -> parseInt(s, minValue, key, isFiltered(properties)), properties);
11831183
}
11841184

1185+
public static Setting<Integer> intSetting(String key, Setting<Integer> fallbackSetting, int minValue, int maxValue,
1186+
Property... properties) {
1187+
return new Setting<>(key, fallbackSetting, (s) -> parseInt(s, minValue, maxValue, key, isFiltered(properties)), properties);
1188+
}
1189+
11851190
public static Setting<Integer> intSetting(String key, Setting<Integer> fallbackSetting, int minValue, Validator<Integer> validator,
11861191
Property... properties) {
11871192
return new Setting<>(new SimpleKey(key), fallbackSetting, fallbackSetting::getRaw,
@@ -1246,7 +1251,7 @@ public static int parseInt(String s, int minValue, int maxValue, String key) {
12461251
return parseInt(s, minValue, maxValue, key, false);
12471252
}
12481253

1249-
static int parseInt(String s, int minValue, int maxValue, String key, boolean isFiltered) {
1254+
public static int parseInt(String s, int minValue, int maxValue, String key, boolean isFiltered) {
12501255
int value = Integer.parseInt(s);
12511256
if (value < minValue) {
12521257
String err = "Failed to parse value" + (isFiltered ? "" : " [" + s + "]") + " for setting [" + key + "] must be >= " + minValue;

‎server/src/main/java/org/elasticsearch/http/HttpTransportSettings.java‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -116,9 +116,9 @@ public final class HttpTransportSettings {
116116
public static final Setting<Boolean> SETTING_HTTP_TCP_KEEP_ALIVE =
117117
boolSetting("http.tcp.keep_alive", NetworkService.TCP_KEEP_ALIVE, Setting.Property.NodeScope);
118118
public static final Setting<Integer> SETTING_HTTP_TCP_KEEP_IDLE =
119-
intSetting("http.tcp.keep_idle", NetworkService.TCP_KEEP_IDLE, -1, Setting.Property.NodeScope);
119+
intSetting("http.tcp.keep_idle", NetworkService.TCP_KEEP_IDLE, -1, 300, Setting.Property.NodeScope);
120120
public static final Setting<Integer> SETTING_HTTP_TCP_KEEP_INTERVAL =
121-
intSetting("http.tcp.keep_interval", NetworkService.TCP_KEEP_INTERVAL, -1, Setting.Property.NodeScope);
121+
intSetting("http.tcp.keep_interval", NetworkService.TCP_KEEP_INTERVAL, -1, 300, Setting.Property.NodeScope);
122122
public static final Setting<Integer> SETTING_HTTP_TCP_KEEP_COUNT =
123123
intSetting("http.tcp.keep_count", NetworkService.TCP_KEEP_COUNT, -1, Setting.Property.NodeScope);
124124
public static final Setting<Boolean> SETTING_HTTP_TCP_REUSE_ADDRESS =

‎server/src/main/java/org/elasticsearch/transport/TransportSettings.java‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -101,15 +101,15 @@ public final class TransportSettings {
101101
fallback(key, OLD_TCP_KEEP_ALIVE_PROFILE, "tcp\\.keep_alive$", "tcp_keep_alive"),
102102
Setting.Property.NodeScope));
103103
public static final Setting<Integer> TCP_KEEP_IDLE =
104-
intSetting("transport.tcp.keep_idle", NetworkService.TCP_KEEP_IDLE, -1, Setting.Property.NodeScope);
104+
intSetting("transport.tcp.keep_idle", NetworkService.TCP_KEEP_IDLE, -1, 300, Setting.Property.NodeScope);
105105
public static final Setting.AffixSetting<Integer> TCP_KEEP_IDLE_PROFILE =
106106
affixKeySetting("transport.profiles.", "tcp.keep_idle",
107-
key -> intSetting(key, TCP_KEEP_IDLE, -1, Setting.Property.NodeScope));
107+
key -> intSetting(key, TCP_KEEP_IDLE, -1, 300, Setting.Property.NodeScope));
108108
public static final Setting<Integer> TCP_KEEP_INTERVAL =
109-
intSetting("transport.tcp.keep_interval", NetworkService.TCP_KEEP_INTERVAL, -1, Setting.Property.NodeScope);
109+
intSetting("transport.tcp.keep_interval", NetworkService.TCP_KEEP_INTERVAL, -1, 300, Setting.Property.NodeScope);
110110
public static final Setting.AffixSetting<Integer> TCP_KEEP_INTERVAL_PROFILE =
111111
affixKeySetting("transport.profiles.", "tcp.keep_interval",
112-
key -> intSetting(key, TCP_KEEP_INTERVAL, -1, Setting.Property.NodeScope));
112+
key -> intSetting(key, TCP_KEEP_INTERVAL, -1, 300, Setting.Property.NodeScope));
113113
public static final Setting<Integer> TCP_KEEP_COUNT =
114114
intSetting("transport.tcp.keep_count", NetworkService.TCP_KEEP_COUNT, -1, Setting.Property.NodeScope);
115115
public static final Setting.AffixSetting<Integer> TCP_KEEP_COUNT_PROFILE =

0 commit comments

Comments
 (0)