Introduce `setDefaultCharacterEncoding()` in `MockHttpServletResponse`

[jhyot]

Since: Spring 5.2 and the deprecation of APPLICATION_JSON_UTF8 Mediatype.

Although the content type 'application/json' can only be encoded in UTF-8 (which is why the explicit encoding was deprecated), MockHttpServletResponse#getCharacterEncoding() returns 'ISO-8859-1'.

This breaks any test code that inspects or uses the getCharacterEncoding() directly. But more unexpectedly, it breaks MockHttpServletResponse#getContentAsString(), because the response is decoded with ISO-8859-1 instead of UTF-8. Explicitly using MockHttpServletResponse#getContentAsString(Charset fallbackCharset) works correctly.

The Javadoc for getContentAsString() is (to me) somewhat ambiguous:

Get the content of the response body as a String, using the charset specified for the response by the application, either through HttpServletResponse methods or through a charset parameter on the Content-Type.

"using the charset specified for the response by the application" could imply UTF-8 for 'application/json', because that must always be the value for JSON. If using this interpretation, then the method breaks its Javadoc contract.

Previously, this was fixed for ContentResultMatchers#json(String) but not for MockHttpServletResponse#getContentAsString() (#23622).
Another report was closed by the submitter without any resolution except using an explicit charset (#23851).

If it is not possible to fix this for JSON specifically, I would appreciate if somehow a default value could be set (instead of always defaulting to ISO-8859-1).

[sbrannen]

I would appreciate if somehow a default value could be set (instead of always defaulting to ISO-8859-1).

Are you aware of the org.springframework.mock.web.MockHttpServletResponse.setCharacterEncoding(String) method?

[jhyot]

@sbrannen Yes I saw the method, but I didn't think of a way to do it by default. I guess I could write a custom org.springframework.test.web.servlet.ResultHandler and set it there. Thanks.

The rest of the ticket is still valid with the somewhat unexpected behaviour.

[sbrannen]

@sbrannen Yes I saw the method, but I didn't think of a way to do it by default. I guess I could write a custom org.springframework.test.web.servlet.ResultHandler and set it there. Thanks.

You could write a custom ResultHandler to set the character encoding for the response, but you would probably be better off setting it as early as possible -- for example, via a custom Filter registered via .addFilter() in the MockMvc builder.

Note, however, that setting the character encoding via setCharacterEncoding() or setContentType() results in ;charset=... being appended to the Content-Type header, and that may be an unacceptable side effect for your use case.

In light of that, I am going to repurpose this issue to introduce support for changing the default character encoding used in MockHttpServletResponse.

The rest of the ticket is still valid with the somewhat unexpected behaviour.

Although I can see how you might have expected getContentAsString() to take into account the Content-Type header, I don't think we should alter the behavior to provide special treatment for application/json to infer UTF-8 as the character encoding since getContentAsString(Charset) was introduced to support use cases where the character encoding used to write the response body is not reflected in the value returned from getCharacterEncoding().

@rstoyanchev, do you have any further input here?