Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

strictly follow DIFFERENTIAL draft spec #1

Open
derhuerst opened this issue Mar 13, 2020 · 5 comments · May be fixed by #2
Open

strictly follow DIFFERENTIAL draft spec #1

derhuerst opened this issue Mar 13, 2020 · 5 comments · May be fixed by #2
Assignees
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@derhuerst
Copy link
Owner

derhuerst commented Mar 13, 2020

We should carefully read the draft DIFFERENTIAL spec and make sure this package follow it. There are already too many homegrown almost-compatible GTFS-RT tools out there, this package shouldn't be another one!

@derhuerst derhuerst added bug Something isn't working help wanted Extra attention is needed labels Mar 13, 2020
@derhuerst
Copy link
Owner Author

derhuerst commented Mar 13, 2020

Some comments from the draft Google doc:

Finally, the consumer should ensure that updates received together in a single FeedMessage become visible to an end user simultaneously. The user should never observe a state where some updates in a FeedMessage have been applied but others have not.

The GTFS-RT reference document states that the FeedEntity.id and FeedEntity.isDeleted fields are only relevant to incremental feeds. However, given the differential semantics outlined above there is not a clear need to explicitly delete RT messages (other than Alerts, see discussion below) as one can always simply provide an update that will supersede them.

TripUpdates: Both scheduled and added trips can be CANCELED by sending another TripUpdate message referencing the same trip ID.
VehiclePositions: When a message is received saying the vehicle is STOPPED_AT the last stop in a trip, or saying that a vehicle has moved on to a different trip, its state with respect to that first trip is effectively removed. However, it might be helpful to provide a means to unambiguously signal that a vehicle has gone out of service.
Alerts. Unlike the other two message types, multiple Alerts may accumulate to the same GTFS entity. Without alert IDs and deletion flags, it would not be possible to remove them in differential mode. We can completely sidestep the problem by simply not using differential Alerts. Alert datasets are generally small and do not benefit greatly from low update latency. They seem to be a good use case for the FULL_DATASET provider push combination.

I think these steps must be implemented for full compatibility:

  • consume FeedMessages instead of FeedEntitys
  • delete TripUpdate entities using DELETED
  • delete VehiclePosition entities using STOPPED_AT
  • support ttl: Infinity
  • lib/entities-store: respect FeedMessage.header.timestamp when setting the expiry timer?

good to know:

  1. Differential vs. Incremental. The reference and specification use both terms: differential and incremental. We should choose only one and use it consistently. These same terms are used in backup systems, and the behavior here seems closer to incremental backups where only individual files that have changed since the last backup (be it incremental, differential, or full) are included. In some sense, GTFS-RT as a whole is “differential” with respect to scheduled GTFS, and the new mode we are discussing is “incremental” with respect to a GTFS-RT FULL_DATASET.

derhuerst added a commit that referenced this issue Oct 7, 2021
derhuerst added a commit that referenced this issue Oct 7, 2021
@derhuerst
Copy link
Owner Author

derhuerst commented Mar 4, 2022

The OTP docs also refer to this lib which implements the OTP-specific "GTFS-RT FeedEntitys over WebSockets" style: https://github.com/OneBusAway/onebusaway-gtfs-realtime-exporter

@derhuerst
Copy link
Owner Author

derhuerst added a commit that referenced this issue Sep 15, 2024
derhuerst added a commit that referenced this issue Sep 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant