Streaming response early disconnect mode

[dmitry-mli]

Summary

Starlette 0.38.4 introduced a bugfix that also introduced a backward incompatible change. This PR keep both the original fix as well as recovers the backward compatibility through parameterization.

#2620 solved issue

#2620 delivered an important fix for BaseHTTPMiddleware which relied on StreamingResponse for chained upstream and downstream communication. Unfortunately, StreamingResponse optimized for early disconnect which resulted into unexpected termination for some of middleware handlers, in case the communication did not manage to propagate through a longer stack (a stack of 4 was enough to reproduce).

#2620 created issue

Change #2620 introduced a backward incompatible change for streaming response from BaseHTTPMiddleware (background tasks were dropped). This impacted software that relied async post-processing.

This PR content

This PR extends StreamingResponse with early_disconnect flag (default True) that controls the early disconnect behavior that is undesirable for the middleware case BaseHTTPMiddleware. The flag is backward compatible for clients of StreamingResponse. The _StreamingResponse is reverted back to inherit from StreamingResponse but this time suppressing the early disconnect mode. Unit tests extended to cover the case.

Checklist

• I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
• I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• [N/A] I've updated the documentation accordingly.

[dmitry-mli]

@Kludex @adriangb please review, thank you! This fixes backward incompatible change made in #2620, details in the description.

@dzhulgakov FYI

[adriangb]

@Kludex another reason we should do #2176

[adriangb]

Thank you for proposing a fix!

Can we go with #2688 instead? It's a much smaller diff / code change.

[dmitry-mli]

Sure thing, approved (with comments)!