Provider response outside of operating hours

[clarityflowers]

Right now, the following is true for the provider /status_changes and /trips endpoints:

For hours in which the provider was not operating the API shall return a 404 Not Found response

However, as a consumer of these endpoints, I've found in practice that it's common for 404 responses to be returned in other situations aside from this one. The most common situation is when data is being requested for a city which doesn't exist or has been removed from the feed, but it also sometimes happens that a bug in the system produces a stray 404 as well.

We want to be able to adjust our ingestion logic when we get these 404 errors, but we can't safely guarantee that we actually are getting the error we think we're getting, which leaves us with the undesirable process of hard-coding checks for each provider's specific and unspecified error messages.

My proposed solution is to define a specific error message string which must also be returned in addition to the 404, something like HOUR NOT IN SERVICE, so that it is completely clear that this is the specific reason that the 404 has been returned.

This would only be a breaking change for providers.

Alternatively the error code could be changed to a less common value (ie, 410, which some operators are already returning under older versions of MDS before this error code was defined). This would be a breaking change for both providers and consumers, so I imagine it is less preferable.

[jiffyclub]

Linking to #437, in which the current language was added and we had some related discussion. To clarify something from there, the 404 status is not meant to be used for short absences in service, like overnight, but only for long-term service interruptions like closing for a season or leaving a market (or before entering a market). For interruptions on the order of hours or days providers should still be return 200 responses with empty arrays of data. Maybe we could add some clarification on that to the provider readme.

I think we've also had some discussion about having a provider API where providers could share the dates they were in operation in a region, which would help with this ambiguity. I can't remember if that was written down or if it only came up on a call. If I had a choice I'd prefer that solution over anything else because it gives us explicit, actionable info.

We deal with the issue of operators sending 404 for hours in which they were operating but nothing happened and have also had to implement operator-specific logic for this. I think the spec is fairly clear that that case should result in 200 responses, but here we are.

[schnuerle]

I think this Issue #628 would be a good solution to this problem.