ClassCastExceptions on several different Graph API endpoints

[veevagoon]

Summary

When executing a variety of different Graph API requests, we've noticed the following ClassCastExceptions occurring:

java.lang.ClassCastException: java.io.BufferedInputStream cannot be cast to com.microsoft.graph.requests.extensions.DriveItemInviteCollectionResponse
        at com.microsoft.graph.requests.extensions.DriveItemInviteCollectionRequest.post(DriveItemInviteCollectionRequest.java:58)

One example call:

IDriveItemInviteCollectionPage permissionsPage = graphClient.drives(driveId)
                .items(itemId)
                .invite(requireSignIn, roles, sendInvitation, message, driveRecipients)
                .buildRequest()
                .select(buildSelectClause(properties))
                .post();

Why I filed a new issue

There was a similar issue filed at #635. Noting that we are also using version 1.6.0. The reason why I filed this new issue is because there's some slight variance in the calls that are affected and I had additional information to share on why they are being affected from debugging through the issue. With this additional information, I wanted to confirm whether updating our library version would solve the issue. We have updated our library to 2.3.2 for newer versions of our application, and I'd like to verify that this would be sufficient for solving the issue since updating the library is non-trivial for our process.

Root Cause

The issue appears to be intermittent and is entirely dependent on what the SDK gets back from making an HTTP Request to the Graph API. The relevant code in the API is in CoreHttpProvider#sendRequestInternal seen below (https://github.com/microsoftgraph/msgraph-sdk-java/blob/1.6.0/src/main/java/com/microsoft/graph/http/CoreHttpProvider.java)

@SuppressWarnings("unchecked")
private <Result, Body, DeserializeType> Result sendRequestInternal(final IHttpRequest request,
		final Class<Result> resultClass,
		final Body serializable,
		final IProgressCallback<Result> progress,
		final IStatefulResponseHandler<Result, DeserializeType> handler)
				throws ClientException {
		...
		in = new BufferedInputStream(response.body().byteStream());

		final Map<String, String> headers = CoreHttpProvider.getResponseHeadersAsMapStringString(response);

		final String contentType = headers.get(Constants.CONTENT_TYPE_HEADER_NAME);
		if (contentType != null && contentType.contains(Constants.JSON_CONTENT_TYPE)) {
			logger.logDebug("Response json");
			return handleJsonResponse(in, CoreHttpProvider.getResponseHeadersAsMapOfStringList(response), resultClass);
		} else {
			logger.logDebug("Response binary");
			isBinaryStreamInput = true;
			//no inspection unchecked
			return (Result) handleBinaryStream(in);
		}
		...
}

Here's the generic code that's used in the SDK for every Graph API request on 1.6.0 (previous version we're using). This code is deciding what kind of format to return the response in based on the "Content-Type" header and the caller of the method. It's deciding between whether to return a JSON object with information about the call made (for most calls), or a BufferedInputStream (for calls that deal with transferring data).

The root cause of the issue is that the "Content-Type" header can come in varying cases (some caps or all lowercase). The Graph API in 1.6.0 is looking for "Content-Type" case-sensitive to determine the content type. When the request returns "content-type", the code cannot find the correct header since it is case-sensitive and defaults to returning the BufferedInputStream. This leads to the exception we've been seeing.

The questions I have for the SDK team are:

• Do we know whether or not this particular "Content-Type" header issue has been resolved in a later version? Based on what I was looking at in the 2.3.2 version of the SDK, it appears the issue would still exist unless the HTTP Response from the Graph API has been sanitized in some way to always guarantee the right case of "Content-Type"
• What has changed in the Graph API to cause this to occur more frequently? As of this week, we've seen all of our calls run into this issue several times more often than in the past (where we've maybe encountered the issue once in our application)

Expected behavior

When performing any Graph API call, we expect the proper contract object to be returned. For the following example:

IDriveItemInviteCollectionPage permissionsPage = graphClient.drives(driveId)
                .items(itemId)
                .invite(requireSignIn, roles, sendInvitation, message, driveRecipients)
                .buildRequest()
                .select(buildSelectClause(properties))
                .post();

We expect a IDriveItemInviteCollectionPage to be returned.

Actual behavior

The following ClassCastException is returned:

java.lang.ClassCastException: java.io.BufferedInputStream cannot be cast to com.microsoft.graph.requests.extensions.DriveItemInviteCollectionResponse
        at com.microsoft.graph.requests.extensions.DriveItemInviteCollectionRequest.post(DriveItemInviteCollectionRequest.java:58)

Steps to reproduce the behavior

The issue is intermittent and cannot be reliably reproduced each call. I see the issue occur 1 in 10 calls that I make using the SDK.
AB#7967

[LeoDorbes]

Hello,
I am encountering the exact same problem on version 1.8.0, and would like to know if upgrading to a newer version will fix the error, and if so, which version will do.

Thanks in advance for your answers.

[katzeX]

Hi Team,

I've remarked a similar exception:

class java.io.BufferedInputStream cannot be cast to class com.microsoft.graph.requests.extensions.DirectoryObjectCollectionResponse (java.io.BufferedInputStream is in module java.base of loader 'bootstrap'; com.microsoft.graph.requests.extensions.DirectoryObjectCollectionResponse is in unnamed module of loader 'app')

At the moment, we are on 1.7.1 and are wondering to upgrade the version. Besides, wanted to ask if the version of microsoft-graph-core is important in this case.

Thanks in advance.

[ivanmaria]

Hi,
I have been facing this issue since past few days. Here is what I have found...

Do we know whether or not this particular "Content-Type" header issue has been resolved in a later version? Based on what I was looking at in the 2.3.2 version of the SDK, it appears the issue would still exist unless the HTTP Response from the Graph API has been sanitized in some way to always guarantee the right case of "Content-Type"

In 2.2.0 version of the SDK, the following code was added to fix the case sensitive issue of Content-Type

public Map<String, String> getResponseHeadersAsMapStringString(final Response response) {
		final Map<String, String> headers = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
		int index = 0;
		Headers responseHeaders = response.headers();
		while (index < responseHeaders.size()) {
			final String headerName = responseHeaders.name(index);
			final String headerValue = responseHeaders.value(index);
			if (headerName == null || headerValue == null) {
				break;
			}
			headers.put(headerName, headerValue);
			index++;
		}
		return headers;
    }

What has changed in the Graph API to cause this to occur more frequently? As of this week, we've seen all of our calls run into this issue several times more often than in the past (where we've maybe encountered the issue once in our application)

+1 Even I am waiting for the answer. What else have changed that might break previous version compatibility?

[baywet]

Hi everyone.
Thanks for reaching our and for the very detailed investigations here.

It looks like the service team has rolled out globally a change on the service that lowercases header names, and that change was previously only deployed to some of the instances, hence why some of you might have seen the issue before and the recent surge. Please keep in mind I'm still trying to confirm that with the service team and this hypothesis is not confirmed yet.

Regardless, http headers names are case insensitive according to the spec and the SDK should handle such differences properly. The code sections that have been shared on this thread do not show any normalization of the headers before making use of those.

I'll get started on a fix right away to unblock you as soon as possible. This fix will come out with version 2.6.1 or 2.7.0 (I'll confirm that later) and not to prior versions. So if you're on older versions of the SDK, please upgrade to the latest version in the meantime.

As for the questions about the core library, we always encourage you to be on the latest version as it might contain fixes for bugs that have a broader impact. You can always know the difference between the version you are currently using and the latest one by looking at the milestones and/or using the releases

[baywet]

Hi everyone,
I've spent the last few hours investigating the issue and I can confirm that version 2.2.0 and up fixes this issue through #481.
Of course we encourage you to upgrade to the latest version as it fixes other bugs you might run into.

Additionally, it looks like the change of casing for the content type header is being rolled out globally.
Instances roles formatted as AGSFE_IN_13 reply with Content-Type (not breaking the older versions of the SDK).
Instances roles formatted as QB1PEPF00000F2C reply with content-type (breaking the older versions of the SDK).
(looking at the x-ms-ags-diagnostic header)

I've noticed a few opportunities for improvements to avoid unnecessary CPU usage which I'll submit a pull request for soon.