Merge branch 'fix/10310/freezeHttpFields' into experiment/jetty-12.0.…

…x/HttpFieldsContainsLast

documentation/jetty-asciidoctor-extensions/src/main/java/org/eclipse/jetty/docs/JettyIncludeExtension.java

@@ -89,9 +89,10 @@ public void process(Document document, PreprocessorReader reader, String target,
         {
             try
             {
+                String jettyVersion = (String)document.getAttribute("project-version");
                 // Document attributes are converted by Asciidoctor to lowercase.
                 Path jettyDocsPath = Path.of((String)document.getAttribute("project-basedir"));
-                Path jettyHome = jettyDocsPath.resolve("../../jetty-home/target/jetty-home").normalize();
+                Path jettyHome = jettyDocsPath.resolve("target/jetty-home-" + jettyVersion).normalize();
 
                 JettyHomeTester jetty = JettyHomeTester.Builder.newInstance()
                     .jettyHome(jettyHome)

documentation/jetty-documentation/pom.xml

@@ -210,6 +210,30 @@
           </execution>
         </executions>
       </plugin>
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-dependency-plugin</artifactId>
+        <executions>
+          <execution>
+            <id>unpack-jetty-home</id>
+            <phase>generate-resources</phase>
+            <goals>
+              <goal>unpack</goal>
+            </goals>
+            <configuration>
+              <artifactItems>
+                <artifactItem>
+                  <groupId>org.eclipse.jetty</groupId>
+                  <artifactId>jetty-home</artifactId>
+                  <version>${project.version}</version>
+                  <type>zip</type>
+                </artifactItem>
+              </artifactItems>
+              <outputDirectory>${project.build.directory}/</outputDirectory>
+            </configuration>
+          </execution>
+        </executions>
+      </plugin>
       <plugin>
         <artifactId>maven-assembly-plugin</artifactId>
         <configuration>
@@ -328,5 +352,12 @@
       <groupId>org.eclipse.jetty.ee10.websocket</groupId>
       <artifactId>jetty-ee10-websocket-jakarta-server</artifactId>
     </dependency>
+    <dependency>
+      <groupId>org.eclipse.jetty</groupId>
+      <artifactId>jetty-home</artifactId>
+      <version>${project.version}</version>
+      <type>zip</type>
+      <optional>true</optional>
+    </dependency>
   </dependencies>
 </project>

documentation/jetty-documentation/src/main/asciidoc/programming-guide/arch-threads.adoc

@@ -178,6 +178,34 @@ Jetty's default thread pool implementation is link:{javadoc-url}/org/eclipse/jet
 
 `QueuedThreadPool` integrates with the xref:pg-arch-bean[Jetty component model], implements `Executor`, provides a `TryExecutor` implementation (discussed in the xref:pg-arch-threads-execution-strategy-adaptive[adaptive execution strategy section]), and supports xref:pg-arch-threads-thread-pool-virtual-threads[virtual threads] (introduced as a preview feature in Java 19 and Java 20, and as an official feature since Java 21).
 
+[[pg-arch-threads-thread-pool-queue]]
+===== Thread Pool Queue
+
+`QueuedThreadPool` uses a `BlockingQueue` to store tasks that will be executed as soon as a thread is available.
+
+It is common, but too simplistic, to think that an upper bound to the thread pool queue is a good way to limit the number of concurrent HTTP requests.
+
+In case of asynchronous servers like Jetty, applications may have more than one thread handling a single request.
+Furthermore, the server implementation may produce a number of tasks that _must_ be run by the thread pool, otherwise the server stops working properly.
+
+Therefore, the "one-thread-per-request" model is too simplistic, and the real model that predicts the number of threads that are necessary is too complicated to produce an accurate value.
+
+For example, a sudden large spike of requests arriving to the server may find the thread pool in an idle state where the number of threads is shrunk to the minimum.
+This will cause many tasks to be queued up, way before an HTTP request is even read from the network.
+Add to this that there could be I/O failures processing requests, which may be submitted as a new task to the thread pool.
+Furthermore, multiplexed protocols like HTTP/2 have a much more complex model (due to xref:pg-server-http2-flow-control[data flow control]) where the implementation must be able to write in order to progress reads (and must be able to read in order to progress writes), possibly causing more tasks to be submitted to the thread pool.
+
+If any of the submitted tasks is rejected because the queue is bounded the server may grind to a halt, because the task _must_ be executed, sometimes _necessarily_ in a different thread.
+
+For these reasons:
+
+IMPORTANT: The thread pool queue must be unbounded.
+
+There are better strategies to limit the number of concurrent requests, discussed in xref:pg-server-http-handler-use-limit[this section].
+
+[[pg-arch-threads-thread-pool-configuration]]
+===== `QueuedThreadPool` configuration
+
 `QueuedThreadPool` can be configured with a `maxThreads` value.
 
 However, some of the Jetty components (such as the xref:pg-arch-io-selector-manager[selectors]) permanently steal threads for their internal use, or rather `QueuedThreadPool` leases some threads to these components.

documentation/jetty-documentation/src/main/asciidoc/programming-guide/server/http/server-http-handler-implement.adoc

@@ -11,7 +11,7 @@
 // ========================================================================
 //
 
-[[pg-server-http-handler-implement]]
+[[pg-server-http-handler-impl]]
 ===== Implementing Handler
 
 The `Handler` API consist fundamentally of just one method:

documentation/jetty-documentation/src/main/asciidoc/programming-guide/server/http/server-http-handler-use.adoc

@@ -42,7 +42,7 @@ Embedded web applications based on the Servlet APIs are described in xref:pg-ser
 Embedded web applications may also require additional features such as access to Jetty specific APIs, or utility features such as redirection from HTTP to HTTPS, support for `gzip` content compression, etc.
 The Jetty Server Libraries provides a number of out-of-the-box ``Handler``s that implement the most common functionalities and are described in the next sections.
 
-[[pg-server-http-handler-use-util-context]]
+[[pg-server-http-handler-use-context]]
 ====== ContextHandler
 
 `ContextHandler` is a `Handler` that represents a _context_ for a web application.
@@ -65,7 +65,7 @@ Server
     └── ShopHandler
 ----
 
-[[pg-server-http-handler-use-util-context-collection]]
+[[pg-server-http-handler-use-context-collection]]
 ====== ContextHandlerCollection
 
 Server applications may need to deploy to Jetty more than one web application.
@@ -76,7 +76,7 @@ However, this has no knowledge of the concept of _context_ and just iterates thr
 A better choice for multiple web application is `ContextHandlerCollection`, that matches a _context_ from either its _context path_ or _virtual host_, without iterating through the ``Handler``s.
 
 If `ContextHandlerCollection` does not find a match, it just returns `false` from its `handle(\...)` method.
-What happens next depends on the `Handler` tree structure: other ``Handler``s may be invoked after `ContextHandlerCollection`, for example `DefaultHandler` (see xref:pg-server-http-handler-use-util-default-handler[this section]).
+What happens next depends on the `Handler` tree structure: other ``Handler``s may be invoked after `ContextHandlerCollection`, for example `DefaultHandler` (see xref:pg-server-http-handler-use-default[this section]).
 Eventually, if no `Handler` returns `true` from their own `handle(\...)` method, then Jetty returns an HTTP `404` response to the client.
 
 [source,java,indent=0]
@@ -96,7 +96,7 @@ Server
         └── RESTHandler
 ----
 
-[[pg-server-http-handler-use-util-resource-handler]]
+[[pg-server-http-handler-use-resource]]
 ====== ResourceHandler -- Static Content
 
 Static content such as images or files (HTML, JavaScript, CSS) can be sent by Jetty very efficiently because Jetty can write the content asynchronously, using direct ``ByteBuffer``s to minimize data copy, and using a memory cache for faster access to the data to send.
@@ -116,7 +116,7 @@ Therefore, the traditional architecture where Nginx/Apache was the front-end ser
 This leads to simpler systems (less components to configure and manage) and more performance (no need to proxy dynamic requests from front-end servers to back-end servers).
 
 NOTE: It is common to use Nginx/Apache as load balancers, or as rewrite/redirect servers.
-We typically recommend link:https://haproxy.org[HAProxy] as load balancer, and Jetty has xref:pg-server-http-handler-use-util-rewrite-handler[rewrite/redirect features] as well.
+We typically recommend link:https://haproxy.org[HAProxy] as load balancer, and Jetty has xref:pg-server-http-handler-use-rewrite[rewrite/redirect features] as well.
 
 This is how you configure a `ResourceHandler` to create a simple file server:
 
@@ -133,9 +133,9 @@ include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPSer
 ----
 
 If the resource is not found, `ResourceHandler` will not return `true` from the `handle(\...)` method, so what happens next depends on the `Handler` tree structure.
-See also xref:pg-server-http-handler-use-util-default-handler[how to use] `DefaultHandler`.
+See also xref:pg-server-http-handler-use-default[how to use] `DefaultHandler`.
 
-[[pg-server-http-handler-use-util-gzip-handler]]
+[[pg-server-http-handler-use-gzip]]
 ====== GzipHandler
 
 `GzipHandler` provides supports for automatic decompression of compressed request content and automatic compression of response content.
@@ -185,7 +185,7 @@ Server
 
 // TODO: does ServletContextHandler really need a special configuration?
 
-[[pg-server-http-handler-use-util-rewrite-handler]]
+[[pg-server-http-handler-use-rewrite]]
 ====== RewriteHandler
 
 `RewriteHandler` provides support for URL rewriting, very similarly to link:https://httpd.apache.org/docs/current/mod/mod_rewrite.html[Apache's mod_rewrite] or link:https://nginx.org/en/docs/http/ngx_http_rewrite_module.html[Nginx rewrite module].
@@ -226,7 +226,7 @@ Server
         └── ContextHandler N
 ----
 
-[[pg-server-http-handler-use-util-stats-handler]]
+[[pg-server-http-handler-use-statistics]]
 ====== StatisticsHandler
 
 `StatisticsHandler` gathers and exposes a number of statistic values related to request processing such as:
@@ -266,7 +266,13 @@ For instance, `StatisticsHandler.MinimumDataRateHandler` can be used to enforce
 include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java[tags=dataRateHandler]
 ----
 
-[[pg-server-http-handler-use-util-secure-handler]]
+[[pg-server-http-handler-use-limit]]
+====== QoSHandler
+TODO
+// TODO: see #10361
+//  Also port documentation from old_docs/extras/qos-filter
+
+[[pg-server-http-handler-use-secured]]
 ====== SecuredRedirectHandler -- Redirect from HTTP to HTTPS
 
 `SecuredRedirectHandler` allows to redirect requests made with the `http` scheme (and therefore to the clear-text port) to the `https` scheme (and therefore to the encrypted port).
@@ -282,7 +288,7 @@ Server applications must configure a `HttpConfiguration` object with the secure
 include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java[tags=securedHandler]
 ----
 
-[[pg-server-http-handler-use-util-default-handler]]
+[[pg-server-http-handler-use-default]]
 ====== DefaultHandler
 
 `DefaultHandler` is a terminal `Handler` that always returns `true` from its `handle(\...)` method and performs the following:

documentation/jetty-documentation/src/main/asciidoc/programming-guide/server/http/server-http-handler.adoc

@@ -47,7 +47,7 @@ Refer to xref:pg-server-http-handler-use[this section] for more information abou
 
 However, in some cases the additional features are not required, or additional constraints on memory footprint, or performance, or just simplicity must be met.
 In these cases, implementing your own `Handler` may be a better solution.
-Refer to xref:pg-server-http-handler-implement[this section] for more information about how to write your own ``Handler``s.
+Refer to xref:pg-server-http-handler-impl[this section] for more information about how to write your own ``Handler``s.
 
 // TODO: document ScopedHandler? Is this really necessary or just an implementation detail that application will never worry about?
 

jetty-core/jetty-server/src/main/java/org/eclipse/jetty/server/handler/DebugHandler.java

@@ -83,7 +83,7 @@ public boolean handle(Request request, Response response, Callback callback) thr
     private void print(String name, String message)
     {
         long now = System.currentTimeMillis();
-        final String d = _date.formatNow(now);
+        final String d = _date.format(now);
         final int ms = (int)(now % 1000);
 
         _print.println(d + (ms > 99 ? "." : (ms > 9 ? ".0" : ".00")) + ms + ":" + name + " " + message);

jetty-core/jetty-tests/jetty-test-client-transports/src/test/java/org/eclipse/jetty/test/client/transport/HttpClientIdleTimeoutTest.java

@@ -135,7 +135,9 @@ public void testIdleServerIdleTimeout(Transport transport) throws Exception
         Thread.sleep(2 * idleTimeout);
 
         // Make sure we can make another request successfully.
-        ContentResponse response2 = client.newRequest(newURI(transport)).send();
+        ContentResponse response2 = client.newRequest(newURI(transport))
+            .timeout(5, TimeUnit.SECONDS)
+            .send();
         assertEquals(HttpStatus.OK_200, response2.getStatus());
     }
 }

jetty-core/jetty-tests/jetty-test-client-transports/src/test/java/org/eclipse/jetty/test/client/transport/TrailersTest.java

@@ -91,6 +91,7 @@ public void succeeded()
             client.newRequest(newURI(transport))
                 .trailersSupplier(() -> requestTrailers)
                 .body(body)
+                .timeout(15, TimeUnit.SECONDS)
                 .send(listener);
 
             // Write the content first, then the trailers.