ntfs_to_netex_france_specs.md

NTFS to Netex France conversion

Version of the implementation: 1.0

Introduction

This document describes how a NTFS is transformed into a Netex profil France feed in Navitia Transit Model.

The resulting ZIP archive is composed of:

• a arrets.xml file containing the description of all stops (Quays and StopPlaces)
• a correspondances.xml file containing all transfers between stops (if no transfer, the file is not created)
• a calendriers.xml containing the services
• a folder reseau_<network_name>_<network_id_md5> for each network containing (keep only alphanumerical characters for network_name)
  • a offre_<line_code>_<line_id_md5>.xml containing the description of routes, and service journeys (keep only alphanumerical characters for line_code)

Schema Validation

To validate the produced XML document, you can use xmllint tool. On a Debian/Ubuntu environment, install the libxml2-utils package.

apt install libxml2-utils

On Windows, you can follow these instructions.

For validation, you need NeTEx schemas.

git clone https://github.com/NeTEx-CEN/NeTEx.git

Then you can validate your file my_file.xml against NeTEx schema with the following command.

xmllint --noout --nonet --huge --schema /path/to/NeTEx/xsd/NeTEx_publication.xsd my_file.xml

Input parameters

A NTFS feed is not sufficient to generate a valid Netex Profil France feed:

• ParticipantRef : The ParticipantRef code must be provided,
• StopProviderCode : The code used to identify the provider in the stop_id generation.

Id of objects

According to the Netex Specification, object identifiers should have a specific formatting. For the other objects, the following format is used: FR:[object type]:[object id]:, with a replacement of : by _ in object_id to avoid conflict between separators.

File headers

Each Netex file starts with a header containing some information generated by the program:

• /PublicationDelivery/@version: x.y:FR-NETEX_nnnn-a.b-c with
  • x.y: the version of Netex XSD of Netex (currently 1.09)
  • nnnn:identification of the profile ("FRANCE", "COMMUN", "ARRET", "LIGNE", "RESEAU", "HORAIRE", "CALENDRIER" or "TARIF")
  • a.b is the profile version (2.1)
  • c is the number of the local implementation (containing only integers and .). See at the top of the document.
• /PublicationDelivery/PublicationTimestamp: generation date of the Netex Feed using ISO8601,
• /PublicationDelivery/ParticipantRef: The value of the corresponding ParticipantRef parameter (see input parameters).

Example:

<?xml version="1.0" encoding="UTF-8"?>
<PublicationDelivery xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
				xsi:schemaLocation="http://www.netex.org.uk/netex"
				xmlns="http://www.netex.org.uk/netex"
				xmlns:xlink="http://www.w3.org/1999/xlink"
				xmlns:ifopt="http://www.ifopt.org.uk/ifopt"
				xmlns:gml="http://www.opengis.net/gml/3.2"
				xmlns:core="http://www.govtalk.gov.uk/core"
				xmlns:siri="http://www.siri.org.uk/siri"
				version="1.09:FR-NETEX-2.1-1.0">
<PublicationTimestamp>2019-07-09T09:32:14Z</PublicationTimestamp>
<ParticipantRef>ABCD</ParticipantRef>
<dataObjects>...</dataObjects>
</PublicationDelivery>

arrets.xml

A stop_area is considered monomodal if all the trips having stop_times referencing any of its stop_points have a physical_mode of the same "Netex mode".

NeTEx Transport Modes

physical_mode_id	TransportMode in Netex
Air	air
Boat	water
Bus	bus
BusRapidTransit	bus
Coach	coach
Ferry	water
Funicular	funicular
LocalTrain	rail
LongDistanceTrain	rail
Metro	metro
RapidTransit	rail
RailShuttle	rail
Shuttle	bus
SuspendedCableCar	cableway
Taxi	ignored
Train	rail
Tramway	tram

Coordinates conversion

The GTFS stop_lon and stop_lat are specified in WGS84. The coordinates are converted to EPSG:2154 (Lambert 93).
Example of Netex declaration:

<gml:pos srsName="EPSG:2154">662233.0 6861519.0</gml:pos>

Quay

Each Quay node corresponds to an NTFS stop_point and is of type ZE (Zone d'embarquement). All Quay elements are grouped into a members element which itself is wrapped into a GeneralFrame.

Netex field	NTFS file	NTFS field	Note
Quay/@id	stops.txt	stop_id	see id formatting
Quay/@version			fixed value any.
Quay/Name	stops.txt	stop_name
Quay/Centroid/Location	stops.txt	stop_lat and stop_lon	see Coordinates conversion; if stop_lat and stop_lon are equals to 0.0, Centroid is absent
Quay/AccessibilityAssessment	stops.txt	equipment_id	This node is present only if the equipment_id is specified. see AccessibilityAssessment below.
Quay/TransportMode			see (2) below
Quay/tariffZones/TariffZoneRef/@ref	stops.txt	fare_zone_id	The fare zone is prefixed by the ParticipantRef prefix with a : separator
Quay/PublicCode	stops.txt	stop_code	This node may not be present if the stop_point has no stop_code.

(2) definition of the TransportMode As a stop_point can be associated to several physical_modes, all the physical_modes need to be mapped to the Netex list (see NeTEx Transport Modes). If more than one Netex mode is associated, use the mode of highest priority (see NTFS specifications).

AccessibilityAssessment

If the stop_point is associated to an equipment, a node AccessibilityAssessment is created and its content is as follow:

Netex field	NTFS file	NTFS field	Note
AccessibilityAssessment/@id	stops.txt		The id is built from the concatenation (joined with _) of the stop_id and the equipment_id. For the rest of the id, use id formatting.
AccessibilityAssessment/@version			fixed value any.
AccessibilityAssessment/MobilityImpairedAccess			see (1) below
AccessibilityAssessment/limitations/AccessibilityLimitation/WheelchairAccess	equipments.txt	wheelchair_boarding	see (2) below
AccessibilityAssessment/limitations/AccessibilityLimitation/AudibleSignsAvailable	equipments.txt	audible_announcement	see (2) below
AccessibilityAssessment/limitations/AccessibilityLimitation/VisualSignsAvailable	equipments.txt	visual_announcement	see (2) below

(1) definition of MobilityImpairedAccess

As stated in NF_Profil NeTEx éléments communs(F) - v2.1.pdf in chapter 5.10: AccessibilityAssessment is optional, but if it's present, MobilityImpairedAccess is mandatory. Therefore, MobilityImpairedAccess should be set to:

• true if all AccessibilityLimitation are set to true
• false if all AccessibilityLimitation are set to false
• partial if some AccessibilityLimitation are set to true
• unknown in other cases

(2) value of limitations

NTFS accessibility value	Netex accessibility value
0 or undefined	undefined
1	true
2	false

StopPlace

StopPlace is of 2 different types:

• LMO (Lieu d'arrêt mono-modal): Mono-modal with a unique name, regrouping Quay
• LMU (Lieu d'arrêt multi-modal): Multi-modal, regrouping multiple StopPlace of type LMO or PM

stop_area can be associated to several physical_mode. All the physical_modes need to be mapped to the Netex list (see NeTEx Transport Modes). For each mode, a StopPlace of type LMO is created. An additionnal LMU is created to regroup them.

Station entrances/exits (stops with location_type = 3) are defined in a multimodal StopPlace to which they belong.

StopPlaceType mapping

The StopPlace/StopPlaceType is defined from its StopPlace/TransportMode.

NeTEx Mode	StopPlaceType
air	Airport
bus	onstreetBus
cableway	liftStation
coach	coachStation
funicular	railStation
metro	metroStation
rail	railStation
tram	tramStation
water	ferryStop

Monomodal StopPlace

Netex field	NTFS file	NTFS field	Note
StopPlace/@id	stops.txt	stop_id	see id formatting
StopPlace/@version			fixed value any.
StopPlace/Name	stops.txt	stop_name
StopPlace/Centroid/Location	stops.txt	stop_lat and stop_lon	see Coordinates conversion; if stop_lat and stop_lon are equals to 0.0, Centroid is absent
StopPlace/ParentSiteRef			link to the corresponding Multimodal StopPlace
StopPlace/TransportMode			use the only NeTEx mode
StopPlace/StopPlaceType			see the section StopPlaceType mapping
StopPlace/quays/QuayRef[]/@ref			see id formatting

Multimodal StopPlace

Netex field	NTFS file	NTFS field	Note
StopPlace/@id	stops.txt	stop_id	see id formatting
StopPlace/@version			fixed value any.
StopPlace/Name	stops.txt	stop_name
StopPlace/Centroid/Location	stops.txt	stop_lat and stop_lon	see Coordinates conversion; if stop_lat and stop_lon are equals to 0.0, Centroid is absent
StopPlace/entrances[]			Link to the station entrances/exits, if present. See StopPlaceEntrance.
StopPlace/TransportMode			use the mode of highest priority (see NTFS specifications)
StopPlace/StopPlaceType			see the section StopPlaceType mapping

StopPlaceEntrance

A StopPlaceEntrance node is created for each entrance/exit (stop with location_type = 3).

Netex field	NTFS file	NTFS field	Note
StopPlaceEntrance/@id	stops.txt	stop_id	see id formatting
StopPlaceEntrance/@version			fixed value any.
StopPlaceEntrance/Name	stops.txt	stop_name
StopPlaceEntrance/Centroid/Location	stops.txt	stop_lat and stop_lon	see Coordinates conversion; if stop_lat and stop_lon are equals to 0.0, Centroid is absent
StopPlaceEntrance/IsEntry			fixed value true
StopPlaceEntrance/IsExit			fixed value true

correspondances.xml

Each connection between two stops in transfers.txt produces a SiteConnection element with the From and To nodes of the connection as well as a WalkTransferDuration node. All SiteConnection elements are grouped in a members element inside a GeneralFrame.

Top level structure

Example:

<?xml version="1.0" encoding="UTF-8"?>
<GeneralFrame id="FR:GeneralFrame:NETEX_RESEAU:" version="any">
	<members>
		<!-- One SiteConnection for each transfer in transfers.txt -->
		<SiteConnection>
			<WalkTransferDuration>
				<DefaultDuration><!-- Walking connecting time --></DefaultDuration>
			</WalkTransferDuration>
			<From>
				<!-- Origin stop of the connection -->
				<StopPlaceRef />
				<QuayRef />
			</From>
			<To>
				<!-- End stop of the connection -->
				<StopPlaceRef />
				<QuayRef />
			</To>
		</SiteConnection>
	</members>
</GeneralFrame>

Netex field	NTFS file	NTFS field	Note
SiteConnection/@id			The id is built from the concatenation (joined with _) of the origin and end stop_id used in the connection. For the rest of the id, use id formatting.
SiteConnection/@version			Fixed value any.
SiteConnection/WalkTransferDuration/DefaultDuration	transfers.txt	real_min_transfer_time	Time is given as a duration (e.g. PT120S for a transfer time of 2 minutes).
SiteConnection/From/StopPlaceRef/@ref			Id of the multimodal StopPlace that contains the origin Quay of the connection. See id formatting.
SiteConnection/From/QuayRef/@ref	transfers.txt	from_stop_id	Id of the origin Quay of the connection. See id formatting.
SiteConnection/To/StopPlaceRef/@ref			Id of the multimodal StopPlace that contains the end Quay of the connection. See id formatting.
SiteConnection/To/QuayRef/@ref	transfers.txt	to_stop_id	Id of the end Quay of the connection. See id formatting.

lignes.xml

Top level structure

Example:

<?xml version="1.0" encoding="UTF-8"?>
<CompositeFrame
			id="FR:CompositeFrame:NETEX_LIGNE:"
			version="any">
	<frames>
		<!-- Repeat a ServiceFrame node for each Network -->
		<ServiceFrame
				id="FR:ServiceFrame:<network_id>:"
				version="any">
			<Network />
		</ServiceFrame>
		<ServiceFrame
				id="FR:ServiceFrame:lines:"
				version="any">
			<lines><!-- One node Line for each Line of the dataset--></lines>
		</ServiceFrame>
		<ResourceFrame
				id="FR:ResourceFrame:operators:"
				version="any">
				<organisations><!-- One node Operator for each company of the dataset --></organisations>
		</ResourceFrame>
	</frames>
</CompositeFrame>

Network

Netex field	NTFS file	NTFS field	Note
Network/@id	networks.txt	network_id	see id formatting
Network/@version			fixed value any
Network/Name	networks.txt	network_name
Network/members/LineRef[]/@ref	lines.txt	line_id	see id formatting

Line

Netex field	NTFS file	NTFS field	Note
Line/@id	lines.txt	line_id	see id formatting
Line/@version			fixed value any
Line/Name	lines.txt	line_name
Line/TransportMode			Refers to the mode with highest priority of the trips associated to the line, see NeTEx Transport Modes and NTFS specifications
Line/PublicCode	lines.txt	line_code	If the code line_code is empty, this node is not created.

Operator

Netex field	NTFS file	NTFS field	Note
Operator/@id	companies.txt	company_id	see id formatting
Operator/@version			fixed value any
Operator/Name	companies.txt	company_name
Operator/ContactDetails/Email	companies.txt	company_mail
Operator/ContactDetails/Phone	companies.txt	company_phone
Operator/ContactDetails/Url	companies.txt	company_url
Operator/OrganisationType			fixed value other

calendriers.xml

Each service_id produce a set of three objects:

• DayType
• DayTypeAssignment
• UicOperatingPeriod

Top level structure

<?xml version="1.0" encoding="UTF-8"?>
<GeneralFrame
		id="FR:GeneralFrame:NETEX_CALENDRIER:"
		version="any">
	<ValidBetween>
		<FromDate>2020-01-01T00:00:00Z</FromDate>
		<ToDate>2020-12-30T23:59:59Z</ToDate>
	</ValidBetween>
	<members>
		<!-- One DayType for each 'service_id' -->
		<DayType />
		<!-- One DayTypeAssignment for each 'service_id' to link
		     a DayType and a UicOperatingPeriod -->
		<DayTypeAssignment />
		<!-- One UicOperatingPeriod for each 'service_id' -->
		<UicOperatingPeriod />
	</members>
</GeneralFrame>

ValidBetween

Netex field	NTFS file	NTFS field	Note
ValidBetween/FromDate	datasets.txt	start_date	minimum from all datasets' start_date; use ISO-8601 date-time format, with a time of 00:00:00
ValidBetween/ToDate	datasets.txt	end_date	maximum from all datasets' end_date; use ISO-8601 date-time format, with a time of 23:59:59

DayType

The DayType must exist even if intentionally left empty (it is referred to by ServiceJourney).

Netex field	NTFS file	NTFS field	Note
DayType/@id	calendar.txt or calendar_dates.txt	service_id	see id formatting
DayType/@version			fixed value any

DayTypeAssignement

Netex field	NTFS file	NTFS field	Note
DayTypeAssignement/@id	calendar.txt	service_id	see id formatting
DayTypeAssignement/@version			fixed value any
DayTypeAssignement/@order			fixed value 0
DayTypeAssignement/OperatingPeriodRef/@ref	calendar.txt	service_id	see id formatting with OperatingPeriod as object type
DayTypeAssignement/DayTypeRef/@ref	calendar.txt	service_id	see id formatting with DayType as object type

UicOperatingPeriod

Netex field	NTFS file	NTFS field	Note
UicOperatingPeriod/@id	calendar.txt	service_id	see id formatting with OperatingPeriod as object type
UicOperatingPeriod/@version			fixed value any
UicOperatingPeriod/FromDate	calendar.txt	start_date	use ISO-8601 date-time format, using T00:00:00Z for the time
UicOperatingPeriod/ValidDayBits			see (1) below

(1) Sequence for ValidDayBits

This is a string sequence of 0 (inactive days) and 1 (active days). The sequence should contains as many 0/1 as there is days in the period. The first character correspond to the first day of the period.

offre_<line_code>_<line_id_md5>.xml

The name of the file is composed of the following fields separated by _:

• fixed value offre
• only the alphanumerical characters of the line_code (for example, a:b/c 1.2.3 becomes abc123)
• the MD5 checksum of the line_id (for example, bob becomes e01096b9ffe3f416157f6ec46c467725)

Top level structure

<?xml version="1.0" encoding="UTF-8"?>
<GeneralFrame
		id="FR:GeneralFrame:NETEX_HORAIRE:"
		version="any">
	<members>
		<!-- One Route for each 'route_id' -->
		<Route />
		<ServiceJourneyPattern />
		<!-- One ScheduledStopPoint for each Stop in a Journey Pattern -->
		<ScheduledStopPoint />
		<!-- One PassengerStopAssignment for each Stop in a Journey Pattern -->
		<PassengerStopAssignment />
		<!-- One ServiceJourney for each 'trip_id' -->
		<ServiceJourney />
	</members>
</GeneralFrame>

Route

Netex field	NTFS file	NTFS field	Note
Route/@id	routes.txt	route_id	see id formatting
Route/@version			fixed value any
Route/Name	routes.txt	route_name
Route/Distance			fixed value 0 (mandatory field in NeTEx France LinkSequence)
Route/LineRef/@ref	routes.txt	line_id	see id formatting with Line as object type
Route/DirectionType	routes.txt	direction_type	see (1) below
Route/pointsInSequence/PointOnRoute[]			see PointOnRoute

(1) Direction Type

Use the following mapping to convert the direction type from NTFS to NeTEx France.

NTFS Direction Type	NeTEx Direction Type
forward	inbound
backward	outbound
inbound	inbound
outbound	outbound
clockwise	clockwise
anticlockwise	anticlockwise

PointOnRoute

For each stop_point of all the vehicle_journey of the current route, a PointOnRoute is created, with a link to a corresponding RoutePoint (see related section).

Netex field	NTFS file	NTFS field	Note
PointOnRoute/@id			see (1) below
PointOnRoute/@version			fixed value any
PointOnRoute/@order			see (2) below
PointOnRoute/RoutePointRef/@ref			see (1) below; see id formatting with RoutePoint as object type for the rest

(1) identifier for PointOnRoute

The identifier is built from the concatenation of the following fields separated by _:

• the route_id
• the @order

For the rest of the identifier, use id formatting.

(2) @order of the PointOnRoute

All the stop_point of the route are inserted with the following rules (@order is auto-incremented from 1).

The vehicle_journeys are processed in order. The ordering is done using the following keys in this order of priority:

1. stop_time[0].stop_id: the stop_point of the first stop_time of the vehicle_journey
2. stop_time[0].departure_time: the departure_time of the first stop_time of the vehicle_journey All the stop_points of the first vehicle_journey are inserted in the list of PointOnRoute. Then, for the second vehicle_journey, insert only the stop_points that are not already present, but in a manner that the order of the stop_points in the vehicle_journey is preserved (priority to the stop_points of the previous vehicle_journeys).

Below is an example with 2 vehicle_journeys, VJ1 (in red) and VJ2 (in blue); stop_points are colored accordingly (mixed colors means they're part of both vehicle_journeys).

Below is the final sequence of PointOnRoute with their corresponding order.

RoutePoint

One RoutePoint is created for each stop_point of all vehicle_journeys of a Route.

Netex field	NTFS file	NTFS field	Note
RoutePoint/@id			see PointOnRoute/@id with RoutePoint as object type
RoutePoint/@version			fixed value any
RoutePoint/Location	stops.txt	stop_lat and stop_lon	see Coordinates conversion; if stop_lat and stop_lon are equals to 0.0, Location is absent

ServiceJourneyPattern

A Journey Pattern is created for each unique sequence of stops in the Vehicle Journeys. Unicity applies to:

• stop_id
• pickup_type
• drop_off_type
• local_zone_id

Netex field	Note
ServiceJourneyPattern/@id	see id formatting using the first trip_id in alphanumerical order
ServiceJourneyPattern/@version	fixed value any
ServiceJourneyPattern/Distance	fixed value 0 (mandatory field in NeTEx France LinkSequence)
ServiceJourneyPattern/RouteRef/@ref	see id formatting with Route as object type
ServiceJourneyPattern/pointsInSequence/StopPointInJourneyPattern[]	see section on StopPointInJourneyPattern

StopPointInJourneyPattern

For each stop of a Journey Pattern, a StopPointInJourneyPattern is created, with a link to a corresponding ScheduledStopPoint (see related section).

Netex field	NTFS file	NTFS field	Note
StopPointInJourneyPattern/@id			see (1) below
StopPointInJourneyPattern/@version			fixed value any
StopPointInJourneyPattern/@order	stop_times.txt	stop_sequence	value of stop_sequence + 1 (because 0 is not a valid value)
StopPointInJourneyPattern/ScheduledStopPointRef/@ref			see (1) below; see id formatting with StopPointInJourneyPattern as object type for the rest
StopPointInJourneyPattern/ForAlighting	stop_times.txt	drop_off_type	0 is set to true, 1 is set to false
StopPointInJourneyPattern/ForBoarding	stop_times.txt	pickup_type	0 is set to true, 1 is set to false

(1) identifier for StopPointInJourneyPattern

The identifier is built from the concatenation of the following fields separated by _:

• the trip_id used in the corresponding ServiceJourneyPattern/@id
• the @order

For the rest of the identifier, use id formatting.

ScheduledStopPoint

One ScheduledStopPoint is created for each Stop Point of a ServiceJourneyPattern.

Netex field	NTFS file	NTFS field	Note
ScheduledStopPoint/@id			see StopPointInJourneyPattern/@id with ScheduledStopPoint as object type
ScheduledStopPoint/@version			fixed value any
ScheduledStopPoint/Location	stops.txt	stop_lat and stop_lon	see Coordinates conversion; if stop_lat and stop_lon are equals to 0.0, Location is absent

PassengerStopAssignment

One PassengerStopAssignment is created for each Stop Point of a ServiceJourneyPattern.

Netex field	NTFS file	NTFS field	Note
PassengerStopAssignment/@id			see StopPointInJourneyPattern/@id with PassengerStopAssignment as object type
PassengerStopAssignment/@version			fixed value any
PassengerStopAssignment/@order	stop_times.txt	stop_sequence	value of stop_sequence + 1 (because 0 is not a valid value)
PassengerStopAssignment/ScheduledStopPointRef/@ref			see StopPointInJourneyPattern/@id with ScheduledStopPoint as object type
PassengerStopAssignment/StopPlaceRef/@ref			refers to the monomodal StopPlace; see the section Identifiers for Quay and StopPlace
PassengerStopAssignment/QuayRef/@ref			see the section Identifiers for Quay and StopPlace

ServiceJourney

Netex field	NTFS file	NTFS field	Note
ServiceJourney/@id	trips.txt	trip_id	see id formatting
ServiceJourney/@version			fixed value any
ServiceJourney/TransportMode	trips.txt	physical_mode_id	use the only NeTEx mode. See (1) below and NeTEx Transport Modes.
ServiceJourney/dayTypes/DayTypeRef/@ref	trips.txt	service_id	see id formatting with DayType as object type
ServiceJourney/JourneyPatternRef/@ref	trips.txt	service_id	see id formatting with DayType as object type
ServiceJourney/OperatorRef/@ref	trips.txt	company_id	see id formatting with Operator as object type
ServiceJourney/passingTimes/TimetabledPassingTime[]	stop_times.txt		see TimetabledPassingTime

(1) definition of TransportMode for ServiceJourney

As stated in NF_Profil NeTEx pour les Horaires(F) - v2.1.pdf in chapter 5.2: TransportMode for ServiceJourney is optional, but is present if it differs from that of its line.

TimetabledPassingTime

Netex field	NTFS file	NTFS field	Note
TimetabledPassingTime/ArrivalTime	stop_times.txt	arrival_time	format as 00:00:00 with a modulo on 24 hours (e.g. 25:00:00 becomes 01:00:00)
TimetabledPassingTime/ArrivalDayOffset	stop_times.txt	arrival_time	number of times 24 hours fits in arrival_time (e.g. for 50:00:00, 24 hours fits 2 times)
TimetabledPassingTime/DepartureTime	stop_times.txt	departure_time	format as 00:00:00 with a modulo on 24 hours (e.g. 25:00:00 becomes 01:00:00)
TimetabledPassingTime/DepartureDayOffset	stop_times.txt	departure_time	number of times 24 hours fits in departure_time (e.g. for 50:00:00, 24 hours fits 2 times)