diff --git a/src/documentation/netex_idfm_to_ntfs.md b/src/documentation/netex_idfm_to_ntfs.md deleted file mode 100644 index 6e3cf8f4a..000000000 --- a/src/documentation/netex_idfm_to_ntfs.md +++ /dev/null @@ -1,381 +0,0 @@ -# Netex IDFM reading specification -## Introduction -This document describes how a Netex feed provided by Ile-de-France Mobilités (IDFM) is read in Navitia Transit model (NTM) -and transformed into an [NTFS feed](https://github.com/CanalTP/ntfs-specification/blob/master/ntfs_fr.md). - -For the sake of simplicity, the NTM properties that are not specified in the source -data are not described below. - -In order to guarantee that the NTFS objects identifiers are unique and stable, each -object id is prefixed with a unique prefix (specified for each datasource), following -the general pattern `:`. This prefix is uniquely defined for all data_sources -to make possible safe data aggregations. - -### Versions of this document: -Version | Date | Modification ---- | --- | --- -1.0 | 2019-09-13 | Initial redaction -1.1 | 2019-09-30 | Adding fare zones, MIP access on lines and stop_points, the use of source coordinates of a stop_area (if available), complementary properties of a line (commercial_mode, line_color, line_text_color, comment on line) -1.2 | 2019-10-07 | Using new source specifications for Netex IDFM lines and stops, reading of `lignes.xml` and `arrets.xml` is reworked (no more complementary codes on stops and lines, associations between objects changed, etc.).
For `stop_points`, lowest level of `Quay` will be used (ZDEp). - -## Input data description -This specification assumes that all the required data (time tables for all the lines, stop points and stop areas, transfers, etc.) are provided in one ZIP archive (aka "FICHIERS OFFRE") described in the private specification documents provided by IDFM: -+ *NT60-A150701-v1.11-BO-STIF_-_Specification_Technique_d_Interface_NeTEx_pour_la_publication_20190624.docx* -+ *ATD-TDI-LZR-069-DINT WS-NETEX-01 v1.0.9_20190805.docx* (section 4.3) -+ *DINT-LIGNE_publication_1.7.3_20180306.docx* (section 4, using the WebService version 2) - -The ZIP archive contains: -- one **arrets.xml** file -- one **lignes.xml** file, containing the description of lines, networks and companies -- one **correspondances.xml** file (not yet described) -- one folder for each operator (or company) containing: - + a **calendriers.xml** file, containing the calendars (or validity patterns) used by trips in files **offre_** - + a **commun.xml** optional file, containing the comments referenced by the operator objects (if needed) - + several files starting with **offre_** describing the time tables of a specific line in each file - -In this document, versions of objects are not handled. The first encountered object description is considered when creating an object. - -Each XML file contains a `PublicationDelivery` node, containing a `dataObjects` node. Descriptions below is considering nodes inside this `dataObjects` node. Furthermore, brackets (`[]`) in an XML tag indicates this tag can appear more than one time. - -## Reading of the "arrets.xml" file into stops.txt file -In the Netex-IDFM feed: -- a `Quay` object defined at the operator level is referenced by a `Quay` object defined by the PTA (IDFM), -- this PTA `Quay` is included in a monomodal `StopPlace`, -- this monomodal `StopPlace` is included in a multimodal `StopPlace`. - -### For stop_areas -`stop_area` objects in the output NTFS are top level `StopPlace` (ie. not included in an other `StopPlace`). -`stop_area` are defined by `CompositeFrame/frames/GeneralFrame[]/StopPlace[]` nodes. Only the `GeneralFrame` with the `TypeOfFrameRef/@ref` attribut set to `FR100:TypeOfFrame:NETEX_ARRET_STIF:` contains the stop_areas. - -A stop_area can therefore be: - -1. a multimodal `StopPlace` -2. a monomodal `StopPlace` without a multimodal `StopPlace` containing it - -A `stop_area` is created for each `StopPlace` not containing a `StopPlace/ParentSiteRef` tag or referencing by `StopPlace/ParentSiteRef/@ref` a `StopPlace` that does not exist in the feed. - -NTFS field | Netex-IDFM element | Mapping rule/Comment ---- | --- | --- -stop_id | StopPlace/@id | This field is prefixed. The technical part of the NeTEx identifier is used. For example, in `FR::multimodalStopPlace:69406:FR1`, the `stop_area` identifier is `:69406` (fourth field with colon `:` separator). In the case of a `stop_area` created from a monomodal `StopPlace`, identifier of the `stop_area` is `:monomodalStopPlace:411396` (third and fourth fields with a colon `:` separator) -stop_name | StopPlace/Name | -location_type | | Fixed value `1` (stop_area) -stop_lat | StopPlace/Centroid/Location | see (1) below -stop_lon | StopPlace/Centroid/Location | see (1) below - -**(1) Definition of stop_lat et stop_lon:** - -The node `StopPlace/Centroid/Location` declares the position (X, Y) of the stop_area with an explicit EPSG always set to 2154. This coordinate will be converted to the NTFS WGS84 projection (EPSG:4326). -Example of Netex declaration: ->662233.0 6861519.0 - -If the Netex feed does not provide coordinates for a stop_area, the stop_lat et stop_lon fields will be set with the coordinate of the centroid of all included stop_points. - -**Complementary object_properties** - -No complementary properties on `stop_area`. - -**Complementary object_codes** - -The `stop_area` has a complementary code `source` with the identifier of the original associated `StopPlace`. - -### For stop_points -`stop_point` objects in the output NTFS are the lowest level `Quays` (those provided by operators). -`stop_points` are defined by `CompositeFrame/frames/GeneralFrame[]/Quay[]` nodes, considering: -- only the `GeneralFrame` with the `TypeOfFrameRef/@ref` attribut set to `FR100:TypeOfFrame:NETEX_ARRET_STIF:` contains the stop_points, -- only the `Quay` nodes having the `Quay/@dataSourceRef` property set to **_anything different_** from `FR1-ARRET_AUTO` - -The `Quay` nodes with a `FR1-ARRET_AUTO` value in `Quay/@dataSourceRef` property are the PTA (IDFM) defined Quays. - -NTFS field | Netex-IDFM element | Mapping rule/Comment ---- | --- | --- -stop_id | Quay/@id | This field is prefixed. The technical part of the NeTEx identifier is used. For example, in `FR::Quay:50117139:FR1`, the `stop_point` identifier is `:50117139` (fourth field with colon `:` separator). -stop_name | Quay/Name | -location_type | | Fixed value `0` (stop_point) -stop_lat | Quay/Centroid/Location | see (1) below -stop_lon | Quay/Centroid/Location | see (1) below -parent_station | | stop_id of the corresponding `stop_area`, see (2) below. -stop_timezone | | fixed value `Europe/Paris` -equipment_id | | This value will be self generated by the connector if an equipment object is necessary. See (2) below. -fare_zone_id | Quay/tariffZones/TariffZoneRef/@ref | If the `tariffZones` does not exist or is empty, this field will be empty. If more than one fare zone are available, the first one will be considered. The fare zone needs to be extract from the content of the `@ref` attribute, using the 3rd field using a `:` separator. If this 3rd field is not an integer, the resulting `fare_zone_id` will be empty. - -**(1) Definition of stop_lat et stop_lon:** - -The node `Quay/Centroid/Location` declares the position (X, Y) of the stop_point with an explicit EPSG always set to 2154. This coordinate will be converted to the NTFS WGS84 projection (EPSG:4326). -Example of Netex declaration: ->662233.0 6861519.0 - -**(2) Definition of the parent_station:** -Linking a stop_area to a stop_point requires to navigate from lower `Quay` to upper `StopPlace` nodes: -- the operator `Quay` node (aka ZDEp) references the PTA `Quay` node (aka ZDEr) with the attribute `Quay/@derivedFromObjectRef`, -- the PTA `Quay` (aka ZDEr) references the monomodal `StopPlace` node (aka ZDL) with the attribute `Quay/ParentZoneRef/@ref` -- the monomodal `StopPlace` node (aka ZDL) references the multimodal `StopPlace` (aka LDA, the actual `stop_area`) with the attribute `StopPlace/ParentSiteRef/@ref` (unless there is no valid multimodal parent, then the monomodal StopPlace is the stop_area). - -If no parent_station is available (neither multimodal nor monomodal), a stop_area will be created using the stop_point properties. - -**Definition of the Accessibility of a stop_point** - -If the `Quay` node corresponding to the stop_point contains a `AccessibilityAssessment/MobilityImpairedAccess` node, an equipment will be generated (in the NTFS **equipments.txt** file) with properties described below. The resulting **equipments.txt** file will contain as few lines as possible to describe all possible sets of values. - -NTFS field | Netex-IDFM element | Mapping rule/Comment ---- | --- | --- -equipment_id | | Auto-incremented value. This field is prefixed. -wheelchair_boarding | Quay/AccessibilityAssessment/MobilityImpairedAccess | If the source value is `false`, `wheelchair_boarding` value is set to `2` (non accessible).
If the source value is `true`, `wheelchair_boarding` value is set to `1` (accessible)
In any other case (other value or missing node) the value is set to `0` (unknown accessibility) -visual_announcement | Quay/AccessibilityAssessment/ limitations/AccessibilityLimitation/VisualSignsAvailable | same rule as `wheelchair_boarding` -audible_announcement | Quay/AccessibilityAssessment/ limitations/AccessibilityLimitation/AudibleSignalsAvailable | same rule as `wheelchair_boarding` - -**Complementary object_codes** - -The `stop_point` has a complementary code `source` with the identifier of the original associated `Quay`. - -## Reading of the "lignes.xml" file - -### networks.txt -`network` objects are provided in the nodes **CompositeFrame/frames/ServiceFrame[]/Network**, each `ServiceFrame` node containing only one `Network` node. - -NTFS field | Netex-IDFM element | Mapping rule/Comment ---- | --- | --- -network_id | Network/@id | This field is prefixed. The technical part of the NeTEx identifier is used. For example, in `FR1:Network:1046:LOC`, the `network` identifier is `:1046` (third field with colon `:` separator). -network_name | Network/Name | -network_timezone | | Fixed value `Europe/Paris`. - -In the case of a line without an associated network, the associated operator is -used as a network. If there is no associated operator, the line is ignored. - -NTFS field | Netex-IDFM element | Mapping rule/Comment ---- | --- | --- -network_id | Operator/@id | This field is prefixed. The technical part of the NeTEx identifier is used, prefixed with `Operator_`. For example, in `FR1:Operator:800:LOC`, the `network` identifier is `:Operator_800` (third field with colon `:` separator). -network_name | Operator/Name | -network_timezone | | Fixed value `Europe/Paris`. - -The `network` has a complementary code `source` with the identifier of the original associated `Network` (or the original associated `Operator`). - -### companies.txt -`companies` (aka operators) are provided in the nodes **CompositeFrame/frames/ResourceFrame/organisations/Operator[]**. There is only one `ResourceFrame` in the file. - -NTFS field | Netex-IDFM element | Mapping rule/Comment ---- | --- | --- -company_id | Operator/@id | This field is prefixed. The technical part of the NeTEx identifier is used. For example, in `FR1:Operator:800:LOC`, the `company` identifier is `:800` (third field with colon `:` separator). -company_name | Operator/Name | - - -### commercial_modes.txt and physical_modes.txt -The transport modes in Netex-IDFM are only defined at the Line level in the node **CompositeFrame/frames/ServiceFrame[]/lines/Line[]/TransportMode**. The only `ServiceFrame` containing the lines has an id property set to `STIF:CODIFLIGNE:ServiceFrame:lineid`. - -Be careful, `physical_mode_id` and `commercial_mode_id` are **not** prefixed. - -Here is the mapping table between a TransportMode value and the corresponding `physical_mode` and `commercial_mode`: - -TransportMode in Netex-IDFM | TransportSubMode in Netex-IDFM | physical_mode_id | physical_mode_name | commercial_mode_id | commercial_mode_name | Note ---- | --- | --- | --- | --- | --- | --- -air | _any_ | Air | Avion | Air | Avion | -bus | _any_ | Bus | Bus | Bus | Bus | -coach | _any_ | Coach | Autocar | Coach | Autocar | -ferry | _any_ | Ferry | Ferry | Ferry | Ferry | -metro | _any_ | Metro | Métro | Metro | Métro | -rail | railShuttle | RailShuttle | Orlyval, CDG VAL | RailShuttle | Orlyval, CDG VAL | -rail | suburbanRailway | LocalTrain | Train Transilien | LocalTrain | Train Transilien | -rail | regionalRail | Train | TER / Intercités | regionalRail | TER | -rail | interregionalRail | Train | TER / Intercités | interregionalRail | Intercités | -rail | local | RapidTransit | RER | RapidTransit | RER | -rail | _others_ | **X** | **X** | **X** | **X** | This modes are ignored. Associated lines (and stop times) are discarded. -trolleyBus | _any_ | Tramway | Tramway | TrolleyBus | TrolleyBus | -tram | _any_ | Tramway | Tramway | Tramway | Tramway | -water | _any_ | Boat | Navette maritime / fluviale | Boat | Navette maritime / fluviale | -cableway | _any_ | Tramway | Tramway | CableWay | CableWay | -funicular | _any_ | Funicular | Funiculaire | Funicular | Funiculaire | -lift | _any_ | Bus | Bus | Bus | Bus | -other | _any_ | Bus | Bus | Bus | Bus | - -All `physical_mode` are enhanced with CO2 emission and fallback modes, following -the documentation in [common.md](common.md#co2-emissions-and-fallback-modes). - -### lines.txt -`lines` are provided in the nodes **CompositeFrame/frames/ServiceFrame[]/lines/Line[]**. The only `ServiceFrame` containing the lines has an id property set to `STIF:CODIFLIGNE:ServiceFrame:lineid`. - -NTFS field | Netex-IDFM element | Mapping rule/Comment ---- | --- | --- -line_id | Line/@id | This field is prefixed. The technical part of the NeTEx identifier is used. For example, in `FR1:Line:C01738:LOC`, the `line` identifier is `:C01738` (third field with colon `:` separator). -network_id | Line/RepresentedByGroupRef/@ref | This field is prefixed. If this attribute or if the referenced network does not exists, this line is not created. -commercial_mode_id | Line/TransportMode | corresponding commercial_mode_id (see mapping above, this field is **not** prefixed). -line_code | Line/PublicCode or Line/ShortName | Use the `PublicCode` value if available and not empty, else the `ShortName` should be used. -line_name | Line/Name | -line_color | Line/Presentation/Colour | If the value is not available or is not a valid hexa RGB, the value `000000` (black) is used. -line_text_color | Line/Presentation/TextColour | If the value is not available or is not a valid hexa RGB, the value `FFFFFF` (white) is used. - -The `line` has a complementary code `source` with the identifier of the original associated `Line`. - -If the node `Line/PrivateCode` is available, the content of this node is added as an `object_code` for this line with `object_system` set at `Netex_PrivateCode`. - -The accessibility of the line, described by `Line/AccessibilityAssessment` node is used at the trip level (see below). - -**Comments on line:** -If at least one `Line/noticeAssignments/NoticeAssignment/NoticeRef` exists and references a valid notice described in one of the **commun.xml** files of the `OFFRE` folders, a link between those notices and the line is described in the NTFS `comment_links.txt` file. - -NTFS field | Netex-IDFM element | Mapping rule/Comment ---- | --- | --- -object_id | Line/@id | This field is prefixed. -object_type | | fixed value `line` -comment_id | Line/noticeAssignments/NoticeAssignment/NoticeRef | This field is prefixed. - - -## Reading of each folder -In a **offre_*.xml** file, 2 **GeneralFrame** are expected in a **CompositeFrame/frames** node: -* one with a `TypeOfFrameRef/@ref` containing the string `NETEX_STRUCTURE` -* one with a `TypeOfFrameRef/@ref` containing the string `NETEX_HORAIRE` - -### routes.txt -`routes` are provided in each **offre_** file in the nodes **CompositeFrame/frames/GeneralFrame[]/members/Route[]**. The `GeneralFrame` to be used is the one with the `@ref` containing `NETEX_STRUCTURE`. - -NTFS field | Netex-IDFM element | Mapping rule/Comment ---- | --- | --- -route_id | Route/@id | This field is prefixed. The source and technical part of the NeTEx identifier are used. For example, in `SNCF:Route:937-C01738-9c749775-ca06-350a-9726-f27b7265ea34:LOC`, the `route` identifier is `:SNCF:937-C01738-9c749775-ca06-350a-9726-f27b7265ea34` (first and third field with colon `:` separator). -line_id | Route/LineRef/@ref | This field is prefixed. -route_name | Route/Name | See [`common.md`](common.md#general-rules) to generate the `name` -direction_type | Route/DirectionType | The value of this field is used without transformation. -destination_id | | The `DirectionRef` of the Route doesn't link to a stop (neither stop_point nor stop_area), thus its value is not used. See [`common.md`](common.md#general-rules) to generate the `destination_id` - -The `route` has a complementary code `source` with the identifier of the original associated `Route`. - -**ServiceJourneyPattern references** - -All ServiceJourneyPattern of a `route` are stored as complementary `object_codes`. ServiceJourneyPattern nodes are listed in the nodes `CompositeFrame/frames/GeneralFrame[]/members/ServiceJourneyPattern[]`. The `GeneralFrame` to be used is the one with the `@ref` containing `NETEX_STRUCTURE`. - -NTFS field | Netex-IDFM element | Mapping rule/Comment ---- | --- | --- -object_type | | fixed value `route` -object_id | ServiceJourneyPattern/RouteRef/@ref | This field is prefixed. -object_system | | fixed value `Netex_ServiceJourneyPattern` -object_code | ServiceJourneyPattern/@id | The value of this field is used without transformation. - - -### trips.txt -`trips` are described in each **offre_*.xml** file. A `trip` combines information of: -- `ServiceJourneyPattern` listed in **CompositeFrame/frames/GeneralFrame[]/members/ServiceJourneyPattern[]** (the `GeneralFrame` is the one with the `@ref` containing `NETEX_STRUCTURE`). -- `ServiceJourney` listed in **CompositeFrame/frames/GeneralFrame[]/members/ServiceJourney[]** (the `GeneralFrame` is the one with the `@ref` containing `NETEX_HORAIRE`), - -The link between those 2 objects is made by `ServiceJourney/JourneyPatternRef/@ref`. - -NTFS field | Netex-IDFM element | Mapping rule/Comment ---- | --- | --- -route_id | ServiceJourney/JourneyPatternRef/@ref | route_id of the Route containing the `JourneyPatternRef` as an object_code. -service_id | | Defined using `ServiceJourney/DayTypeRef`, see (calendar.txt and calendar_dates.txt)[] -trip_id | ServiceJourney/@id | This field is prefixed. The source and technical part of the NeTEx identifier are used. For example, in `STRAV:ServiceJourney:200-C00573-5897063:LOC`, the `trip` identifier is `:STRAV:200-C00573-5897063` (first and third field with colon `:` separator). -trip_headsign | ServiceJourneyPattern/DestinationDisplayRef | Content of the `DestinationDisplay/FrontText` node. If not available, the name of the `stop_point` of the last `stop_time` is used. -trip_short_name | ServiceJourneyPattern/DestinationDisplayRef | Content of the `DestinationDisplay/PublicCode` node. If not available, this field is empty. -company_id | ServiceJourney/OperatorRef | if `ServiceJourney/OperatorRef` is not defined, use `Line/OperatorRef` in [*lines.xml*](#linestxt) file. This field is prefixed. -physical_mode_id | Line/TransportMode | see physical_modes definition (this field is **not** prefixed) -trip_property_id | | see [trip_properties.txt](#trip_propertiestxt) - -The `trip` has a complementary code `source` with the identifier of the original associated `ServiceJourney`. - -**comment_links for a trip** - -If one (or more) `noticeAssignments/NoticeAssignment/NoticeRef` is available in the `ServiceJourney`, a `comment_link` will be created as follow in the `comment_links.txt` file. - -NTFS field | Netex-IDFM element | Mapping rule/Comment ---- | --- | --- -object_id | ServiceJourney/@id | This field is prefixed. -object_type | | Fixed value `trip` -comment_id | noticeAssignments/NoticeAssignment/NoticeRef | This field is prefixed. - -### trip_properties.txt -In the Line declaration (cf. `lines.txt`), if a node `Line/AccessibilityAssessment` exists, a `trip_property` will be specified for all the trips of the line. - -NTFS field | Netex-IDFM element | Mapping rule/Comment ---- | --- | --- -trip_property_id | Line/AccessibilityAssessment/@id | The id of the line is used to create this object. This field is prefixed. -wheelchair_accessible | Line/AccessibilityAssessment/MobilityImpairedAccess | See (1) below. -visual_announcement | Line/AccessibilityAssessment/limitations/AccessibilityLimitation/VisualSignsAvailable | See (1) below. -audible_announcement | Line/AccessibilityAssessment/limitations/AccessibilityLimitation/AudibleSignalsAvailable | See (1) below. - -(1) setting accessibility value: -* If source value is `false`, then this property is set to `2` (not accessible), -* If source value is `true`, then this property is set to `1` (accessible), -* else this property is set to `0` (unknown) - - -### stop_times.txt -`stop_times` of a trip are listed in the `ServiceJourney/passingTimes/TimetabledPassingTime[]` nodes. They are positioned in the same order as the `ServiceJourneyPattern/pointsInSequence` corresponding to the ServiceJourney. - -NTFS field | Netex-IDFM element | Mapping rule/Comment ---- | --- | --- -trip_id | ServiceJourney/@id | This field is prefixed. -stop_sequence | | Auto-incremented field starting with `0` for the first stop_time -stop_id | | See (1) below -arrival_time | TimetabledPassingTime/ArrivalTime | If `TimetabledPassingTime/DepartureDayOffset` value is >0, arrival_time is incremented 24 hours for each day offset (value will be greater than 24:00 for the stop_times on the next day). See (2) for hours passing midnight. -departure_time | TimetabledPassingTime/DepartureTime | If `TimetabledPassingTime/DepartureDayOffset` value is >0, departure_time is incremented 24 hours for each day offset (value will be greater than 24:00 for the stop_times on the next day). -boarding_duration | | Fixed value `0` -alighting_duration | | Fixed value `0` -pickup_type | | See (3) below -drop_off_type | | See (4) below -local_zone_id | | See (4) below - - -(1) Definition of the stop_id of a stop_time: - -1. Find the `ServiceJourneyPattern` referenced by `ServiceJourney/JourneyPatternRef/@ref` of the `TimetabledPassingTime`. -2. The `ServiceJourneyPattern/pointsInSequence/StopPointInJourneyPattern` node of the same position as the stop_time is used. -3. The `StopPointInJourneyPattern/ScheduledStopPointRef/@ref` attribute is searched in the `PassengerStopAssignment/ScheduledStopPointRef/@ref` attribute of all the `PassengerStopAssignment` nodes of the file -4. The `PassengerStopAssignment/QuayRef/@ref` is the stop_id of the stop_point (with a prefix). - -(2) Passing midnight - -When a trip stops around midnight (arriving at stop before midnight and departing after midnight), `TimetabledPassingTime/ArrivalTime` is greater than `TimetabledPassingTime/DepartureTime`. -In this particular situation, `arrival_time` is incremented with 1 day less than the `departure_time`. - -Eg: `TimetabledPassingTime/ArrivalTime` = "23:50:00", `TimetabledPassingTime/DepartureTime` = "00:10:00", `TimetabledPassingTime/DepartureDayOffset` = 1 -The departure is the next day so arrival_time = "23:50:00" and departure_time = "24:10:00" - -(3) Definition of pickup_type and drop_off_type: - -In the `ServiceJourneyPattern/pointsInSequence/StopPointInJourneyPattern` corresponding to this `stop_time` (see `(1)`): -* if the `ForBoarding` node is existing and with a `false` value, `pickup_type` is set to "1" (no boarding) -* else `pickup_type` is set to "0" (regular boarding) - -`drop_off_type` is set using the same method and using the `ForAlighting` node. - -(4) Definition of local_zone_id: - -The declaration of those zones is made in the nodes `RoutingConstraintZone` of the **offre_*.xml** file. -The `local_zone_id` is specified with an auto-incremented integer. Each `RoutingConstraintZone/@id` is associated with a new integer. -This `RoutingConstraintZone` contains a list of `ScheduledStopPointRef` (in `RoutingConstraintZone/members/ScheduledStopPointRef/@ref`). If a stop_time corresponds to one of those `ScheduledStopPointRef`, the `local_zone_id` is set to the associated integer. - -### calendar.txt and calendar_dates.txt -Active days of `trips` are decribed in **calendriers.xml** of each folder in the `GeneralFrame/members/DayType` node (with the use of `DayTypeAssignment` and `OperatingPeriod`). - -A `trip` references one or several `DayType` in `ServiceJourney/dayTypes/DayTypeRef/@ref`. -The `service_id` property of the calendar in the NTFS is specified by an auto-incremented integer (this field is prefixed). - -Active dates of the calendar are specified by: -- `DayType` nodes describing the active days of a week - + days of the week are listed in `DayType/properties/PropertyOfDay[]/DaysOfWeek` nodes. - + Expected values MUST be one of `Monday`, `Tuesday`, `Wednesday`, `Thursday`, `Friday`, `Saturday`, `Sunday`. - + `DayType/properties` node is optional, a `DayType` node may not have any active days of a week. -- `OperatingPeriod` nodes describing periods (basically a beginning date and an end date) referenced in `DayTypeAssignment` -- `DayTypeAssignment` nodes with 2 possible uses (they are applied in the order they appear in the file): - + Activate or deactivate a specific day on a DayType (with the nodes `IsAvailable` and `Date`) - + the node `IsAvailable` is optional. If not present the default value is `true`. - + Apply the active days of the referenced DayType on an `OperatingPeriod` - -All resulting calendars are to be restricted between `ValidBetween/FromDate` and `ValidBetween/ToDate` specified at the top level of the file in `GeneralFrame`. - -Active dates are read in the following order: -- dates from operating periods -- dates added (`DayTypeAssignment/IsAvailable` = true) -- dates removed (`DayTypeAssignment/IsAvailable` = false) - -Be careful: Definition of calendars and exceptions in calendar_dates may not be the same definition as the one in the Netex-IDFM files, but the resulting active dates will be the same. - -Here is 3 possible modelizations in Netex-IDFM of a calendar running from 2016-07-01 to 2016-07-31 from monday to saturday except on 2016-07-14: -![](./netex_idfm_to_ntfs_calendars.png "Definition of calendars in Netex-IDFM specs") - -### comments.txt -`comments` are provided in **commun.xml** file in the nodes **GeneralFrame/members/Notice[]**. - -NTFS field | Netex IDFM element | Mapping rule/Comment ---- | --- | --- -comment_id | *Notice/@id* | This field is prefixed. -comment_name | *Notice/Text* | - -### transfers.txt -Transfers are not yet provided in the Netex IDFM Data feed. Transfers need to be generated afterward to provide accurate trip planning. diff --git a/src/documentation/netex_idfm_to_ntfs_calendars.png b/src/documentation/netex_idfm_to_ntfs_calendars.png deleted file mode 100644 index a53bbc4a1..000000000 Binary files a/src/documentation/netex_idfm_to_ntfs_calendars.png and /dev/null differ diff --git a/src/documentation/nl_hellogo_fares_to_ntm_fare_specs.md b/src/documentation/nl_hellogo_fares_to_ntm_fare_specs.md deleted file mode 100644 index 55ad5c3b1..000000000 --- a/src/documentation/nl_hellogo_fares_to_ntm_fare_specs.md +++ /dev/null @@ -1,88 +0,0 @@ -# HelloGo fares reading specification -## Introduction -This document describes how fares provided by Keolis Netherlands are read in Navitia Transit Model (NTM). - -## Input data description -The fare data expected at the input of the connector comes in the form of XML files based on [Netex fare exchange format](http://www.normes-donnees-tc.org/wp-content/uploads/2014/07/BNTRA-CN03-GT7_N0064_prCEN_TS_278330_FV_E-part_3-v9-1.pdf). Each file is composed of 3 basic elements: -- a *ResourceFrame* specifying the data owner and the validity period of the provided data -- a *ServiceFrame* specifying the network, the lines and the stops to which the fare structure is applied -- one or more *FareFrame*s specifying the fare structures and the underlying conditions. - -The supported fare structures depend on origin-destination (OD) stop pairs in two ways: -- the ticket price between an origin and a destination is directly specified (*DirectPriceMatrix*) -- the price for a fare distance unit is specified, as well as the fare distance between an origin and a destination (*UnitPrice* & *DistanceMatrix*). The ticket price between the origin and destination stops is computed by multiplying the fare distance by the fare distance unit. In this case, one and only one frame of type *UnitPrice* is expected: if none or more than two are provided, the file is considered invalid and should be discarded. - -## Connector description -Each *FareFrame* specified in the input fare data corresponds to several `Tickets` in NTM (as many as the elements of the *DistanceMatrix* and *DirectPriceMatrix*). For each *DistanceMatrixElement*, one `Ticket` object is created specifying the associated line and the origin/destination stops. - -The NTM properties that are not specified in the source data are not described below. - -### Ticket -NTM Property | Source frame | Source element | Notes/Mapping rule ---- | --- | --- | --- -ticket_id | *FareFrame* | *DistanceMatrixElement{id}* | -ticket_name | | | Fixed value `Ticket Origin-Destination`. - -### Ticket_price -NTM Property | Source frame | Source element | Notes/Mapping rule ---- | --- | --- | --- -ticket_id | | | Id of the `Ticket` to which this `Ticket_price` is applied. -ticket_price | | | See the mapping rule below. -ticket_currency | *FareFrame* | *FrameDefaults/DefaultCurrency* | -ticket_validity_start | *ResourceFrame* | *versions/Version/StartDate* | -ticket_validity_end | *ResourceFrame* | *versions/Version/EndDate* | - -#### Computing the ticket price -The ticket price is calculated by adding the boarding fee to the price specified for the origin-destination pair: -- The boarding fee is equal to the value of *FareFrame/EntranceRateWrtCurrency*. -- The OD price is calculated differently based on the *FareStructure* type: - - if the *FareStructure* type is a *DirectPriceMatrix*, the value of *DistanceMatrixElementPrice/Amount* is multiplied by the value of *DistanceMatrixElementPrice/Units*. - - if the *FareStructure* type is a *UnitPrice*, the value of *DistanceMatrixElementPrice/Distance* is multiplied by the value of *GeographicalIntervalPrice/Amount* and then by the value of *GeographicalIntervalPrice/Units*. - -If *FareFrame/RoundingWrtCurrencyRule* is specified, a rounding rule for the specified `currency_type` is applied to the computed ticket price. For example, if the value is set to `0.01` for the currency `EUR`, then the ticket price is rounded to the nearest euro cent. - -If the computed ticket price exceeds the value of *FareFrame/CappingWrtCurrencyRule*, then the latter is taken into account. - -### Ticket_use -NTM property | Source frame | Source element | Notes/Mapping rule ---- | --- | --- | --- -ticket_id | | | Id of the `Ticket` to which this `Ticket_use` is applied. -ticket_use_id | *FareFrame* | *DistanceMatrixElement{id}* | The id is prefixed with `TU:`. -max_transfers | | | Fixed value `0`. -boarding_time_limit | | | This field is explicitly left empty. -alighting_time_limit | | | This field is explicitly left empty. - -### Ticket_use_perimeter -NTM property | Source frame | Source element | Notes/Mapping rule ---- | --- | --- | --- -ticket_use_id | | | Id of the `Ticket_use` to which this `Ticket_use_perimeter` is applied. -object_type | | | Fixed value `line`. -object_id | *FareFrame* | *TriggerObjectRef{ref}* | This value references the associated line in the *ServiceFrame*, for a *TriggerObjectRef{nameOfRefClass}* matching the value `Line`. See the mapping rule below. -perimeter_action | | | Fixed value `1`. - -### Setting the object_id in the Ticket Use Perimeter -Finding the line in the NTFS to which a fare is applied is not straightforward. - -The *TriggerObjectRef{ref}* whose value of *nameOfRefClass* equals to `Line` indicates the line used in a *DistanceMatrix* (and therefore associated with all the enclosed elements). This line points to a *Line* in the *ServiceFrame* with a *KeyValue* element. The *object_id* references the *line_id* found in the NTFS (ignoring the NTFS applied prefix) that matches the *Value* of the *Key* equal to `KV1PlanningLijnNummer`. - -### Ticket_use_restriction -NTM property | Source frame | Source element | Notes/Mapping rule ---- | --- | --- | --- -ticket_use_id | | | Id of the `Ticket_use` to which this `Ticket_use_restriction` is applied. -restriction_type | | | Fixed value `OD`. -use_origin | *FareFrame* | *DistanceMatrixElement/StartStopPointRef{ref}* | This field points to a stop_area found in the NTFS. See the mapping rule below. -use_destination | *FareFrame* | *DistanceMatrixElement/EndStopPointRef{ref}* | This field points to a stop_area found in the NTFS. See the mapping rule below. - -#### Setting the use_origin and use_destination in the Ticket Use Restriction -Finding the right stops in the NTFS to which a fare is applied is not straightforward. - -The stops in the *FareFrame* point to the *ScheduledStopPoint*s in the *ServiceFrame*. The *ScheduledStopPoint*s are composed of *PointProjection*s that are referenced in the NTFS as stop_points. - -The *use_origin* refers to the stop_area found in the NTFS that has an associated stop_point whose *stop_id* (ignoring the NTFS applied prefix) matches the value of *ProjectedPointRef{ref}* (without the network prefix, if any) of the *ScheduledStopPoint* in the *ServiceFrame* having the *id* referenced by *StartStopPointRef{ref}* in the *DistanceMatrixElement*. - -The *use_destination* refers to the stop_area found in the NTFS that has an associated stop_point whose *stop_id* (ignoring the NTFS applied prefix) matches the value of *ProjectedPointRef{ref}* (without the network prefix, if any) of the *ScheduledStopPoint* in the *ServiceFrame* having the *id* referenced by *EndStopPointRef{ref}* in the *DistanceMatrixElement*. - -If multiple *ProjectedPointRef{ref}* are found for *use_origin*, one Ticket Use Restriction will be created for each possible combination of *use_origin* and *use_destination*. Same applies if multiple *ProjectedPointRef{ref}* are found for *use_destination* (all possible combinations). - -If no matching is found for the origin or the destination stop, then the stop is ignored and the corresponding `Ticket` is discarded. - diff --git a/src/documentation/ntfs_fare_v2_to_v1_specs.md b/src/documentation/ntfs_fare_v2_to_v1_specs.md deleted file mode 100644 index a9de2fc04..000000000 --- a/src/documentation/ntfs_fare_v2_to_v1_specs.md +++ /dev/null @@ -1,143 +0,0 @@ -# Converting NTFS fares from V2 to V1 model -## Introduction - -This document describes how [NTFS fare model](https://github.com/CanalTP/ntfs-specification/blob/master/ntfs_fare_extension.md) are converted to the [deprecated fare model](https://github.com/CanalTP/ntfs-specification/blob/master/ntfs_fare_extension_fr_deprecated.md). - -This conversion is only used for a limited timeframe until main navitia product can read the new model. - -The conversion will be illustrated in two steps on the following example. -On the first step, we will consider the case where there is nothing in ticket_use_restrictions.txt. -The second step will take into account ticket_use_restrictions.txt. - -### Data Example - -tickets.txt - -ticket_id | ticket_name | ticket_comment - --- | --- | ---- -my_ticket_id | My Ticket Name | My Ticket Comment - -ticket_prices.txt - -ticket_id | ticket_price| ticket_currency| ticket_validity_start| ticket_validity_end ---- | --- | --- | --- | ---- -my_ticket_id| 1.13 | EUR | 20190101 | 20191231 - - -ticket_uses.txt - -ticket_use_id | ticket_id | max_transfers | boarding_time_limit | alighting_time_limit ---- | --- | --- | --- | ---- -my_use_id | my_ticket_id | 2 | 60 | 90 - - -ticket_use_perimeters.txt - -ticket_use_id | object_type | object_id | perimeter_action - --- | --- | --- | --- -my_use_id | network | my_network | 1 -my_use_id | line | my_line | 1 -my_use_id | line | excluded_line | 2 - - -### Price conversion - -The above example will yield the following prices.csv file : - -prices.csv - -clef ticket | debut validite | fin validite | prix | nom | | commentaire | devise - --- | --- | --- | --- | --- | --- | ---- | --- - my_use_id | 20190101 | 20200101 | 113 | My Ticket Name | | My Ticket Comment | centime - - -There are several things to notice : -- the identifier for the ticket in prices.csv is the *ticket_use_id* and not the ticket_id. Indeed, the NTFS fare model - allows for multiple ticket_use_id associated with the same ticket_id. Hence, to disambiguate, we will use the ticket_use_id - as the reference identifier throughout the conversion -- the end of validity date is incremented by one day in prices.csv compared to ticket_prices.txt. This is because the deprecated - fare model, the end of validity date is interpreted as "the earliest unauthorized day", whereas the NTFS fare model specify it - as "the latest authorized day" -- the price value is converted to "centimes" because the NTFS fare model expect an integer value. Consequently, we support only prices in EUR - in the NTFS fare model. Tickets with a price not given in EUR will be throwed away during the conversion. -- if no (valid) price is found for a ticket it will be skipped / omitted - - -### Without ticket_use_restrictions - -#### Punching a new ticket - -To model our example, we first add two lines to fares.csv in order to allow to punch the ticket. - -fares.csv - -| avant changement | apres changement | debut trajet | fin trajet | condition globale | clef ticket | -| --- | --- | --- | --- | --- | --- | -| | network=my_network | line!=excluded_line & duration<61 & nb_changes<3 | duration<91 | | my_use_id | -| | line=my_line | line!=excluded_line & duration<61 & nb_changes<3 | duration<91 | | my_use_id | - -Each line allows entering one of the two included perimeters : the network my_network and the line my_line. -Note that the exluded perimeter (excluded line) is added as a condition "debut trajet". -The condition "debut trajet" also encodes the constraints on the boarding duration and the number of transfers. -The condition "fin trajet" encodes the constraint on the alighting duration. -The value my_use_id in the "clef ticket" field means that we will punch (and thus pay for) a ticket my_use_id when using these transitions. - -#### Transfers with a punched ticket - -Now that we allowed punching a new ticket, we need to allow transfers with a ticket that has been punched. -This is the purpose of the following lines added to fares.csv - -fares.csv (continued) - -| avant changement | apres changement | debut trajet | fin trajet | condition globale | clef ticket | -| --- | --- | --- | --- | --- | --- | -| network=my_network | network=my_network | ticket=my_use_id & line!=excluded_line & duration<61 & nb_changes<3 | duration<91 | | | -| line=my_line | line=my_line | ticket=my_use_id & line!=excluded_line & duration<61 & nb_changes<3 | duration<91 | | | -| network=my_network | line=my_line | ticket=my_use_id & line!=excluded_line & duration<61 & nb_changes<3 | duration<91 | | | -| line=my_line | network=my_network | ticket=my_use_id & line!=excluded_line & duration<61 & nb_changes<3 | duration<91 | | | - -We add a line for each pair of included perimeters, so as to allow transfers between all perimeters. -We add a constraint "ticket=my_use_id" so as to allow these transfers only if a ticket my_use_id has already been punched (and not another one). -Note that these "transfers" lines will not be added when the maximum number of changes allowed is zero. - - -### With ticket_use_restrictions - -We now enrich our example with the following data in ticket_use_restrictions. - -ticket_use_restrictions.txt - -ticket_use_id | restriction_type | use_origin | use_destination - --- | --- | --- | --- -my_use_id | OD | my_origin | my_destination -my_use_id | zone | my_zone | my_zone - -This means that we can use the ticket my_use_id either between the stop areas my_origin and my_destination or between two stops in the zone my_zone, while satisfying the other constraints of the ticket. -In order to model this, we will put in fares.csv, for each restriction, a copy of the lines described in [the previous section](#without-ticketuserestrictions), -enhanced with the extra constraints corresponding to the restriction. - - -| avant changement | apres changement | debut trajet | fin trajet | condition globale | clef ticket | -| --- | --- | --- | --- | --- | --- | -| | network=my_network | line!=excluded_line & duration<61 & nb_changes<3 & stoparea = my_origin | duration<91 & stoparea = my_destination | | my_use_id | -| | line=my_line | line!=excluded_line & duration<61 & nb_changes<3 & stoparea = my_origin | duration<91 & stoparea = my_destination | | my_use_id | -| network=my_network | network=my_network | ticket=my_use_id & line!=excluded_line & duration<61 & nb_changes<3 & stoparea = my_origin | duration<91 & stoparea = my_destination | | | -| line=my_line | line=my_line | ticket=my_use_id & line!=excluded_line & duration<61 & nb_changes<3 & stoparea = my_origin | duration<91 & stoparea = my_destination | | | -| network=my_network | line=my_line | ticket=my_use_id & line!=excluded_line & duration<61 & nb_changes<3 & stoparea = my_origin | duration<91 & stoparea = my_destination | | | -| line=my_line | network=my_network | ticket=my_use_id & line!=excluded_line & duration<61 & nb_changes<3 & stoparea = my_origin | duration<91 & stoparea = my_destination | | | -| | network=my_network | line!=excluded_line & duration<61 & nb_changes<3 & zone = my_zone | duration<91 & zone = my_zone | | my_use_id | -| | line=my_line | line!=excluded_line & duration<61 & nb_changes<3 & zone = my_zone | duration<91 & zone = my_zone | | my_use_id | -| network=my_network | network=my_network | ticket=my_use_id & line!=excluded_line & duration<61 & nb_changes<3 & zone = my_zone | duration<91 & zone = my_zone | | | -| line=my_line | line=my_line | ticket=my_use_id & line!=excluded_line & duration<61 & nb_changes<3 & zone = my_zone | duration<91 & zone = my_zone | | | -| network=my_network | line=my_line | ticket=my_use_id & line!=excluded_line & duration<61 & nb_changes<3 & zone = my_zone | duration<91 & zone = my_zone | | | -| line=my_line | network=my_network | ticket=my_use_id & line!=excluded_line & duration<61 & nb_changes<3 & zone = my_zone | duration<91 & zone = my_zone | | | - - - -## Shortcomings -Do not put too many OD tickets, or the fare engine performance may noticeably decrease. -Indeed, the fare engine loops over all lines of fares.csv -at least once for each public transport section of each journey computed. -This performance hit could be avoided by using instead the od_fares.csv file, but this prevents enforcing other use cases mandated by -the NTFS fare format (in particular, it is not possible to restrict an od_fares to a specific line or network). -