Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update GTFS parser according to the specifications #398

Merged
merged 63 commits into from
Oct 16, 2019
Merged
Changes from 1 commit
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
c72eb1f
Add some logging in the sanitizer
woshilapin Oct 4, 2019
e843e2c
Add 'source' object code for 'stop_point'
woshilapin Oct 7, 2019
fd463f6
Add 'gtfs_stop_code' in the specification
woshilapin Oct 8, 2019
0be2ab5
Add prefix to Frequency
woshilapin Oct 8, 2019
f54e322
read / write pathways
patochectp Sep 12, 2019
48863ff
read / write levels.txt
patochectp Sep 13, 2019
b07a1f5
manage level_id on stops
patochectp Sep 13, 2019
819cf06
update test
patochectp Sep 13, 2019
24b5f33
stops.txt : read / write StopLocation
patochectp Sep 18, 2019
24599a9
fix bug on id
patochectp Sep 19, 2019
c9009a5
refactoring
patochectp Sep 25, 2019
8123147
update test
patochectp Sep 25, 2019
416debd
gtfs parser refactoring
patochectp Oct 2, 2019
0033f9b
ntfs parser refactoring
patochectp Oct 3, 2019
7cbee69
add test
patochectp Oct 7, 2019
a75cb17
consideration of comments
patochectp Oct 8, 2019
b2d2d2a
fix fmt error
patochectp Oct 9, 2019
49c1acf
Merge pull request #404 from woshilapin/prefix-frequencies
mergify[bot] Oct 9, 2019
5231aaa
Merge branch 'master' into gtfs_stop_code
mergify[bot] Oct 9, 2019
43cc7da
Merge pull request #403 from woshilapin/gtfs_stop_code
mergify[bot] Oct 9, 2019
79d12ce
Merge branch 'master' into stop-point-source
mergify[bot] Oct 9, 2019
b0fc2d2
Merge pull request #401 from woshilapin/stop-point-source
mergify[bot] Oct 9, 2019
7a4492b
Merge branch 'master' into sanitize-logs
mergify[bot] Oct 10, 2019
b9c8a4d
Merge pull request #399 from woshilapin/sanitize-logs
mergify[bot] Oct 10, 2019
ceee671
Bump version 0.9.0
Oct 10, 2019
fa5f96d
Provide a 'only_child' blanket implementation
woshilapin Oct 11, 2019
307195f
Transform Into implementation into From
woshilapin Oct 11, 2019
9a00759
Tests stops without geolocation and pathways with stairs
woshilapin Oct 11, 2019
0f76596
upgrade minidom
Oct 11, 2019
c3d37e4
[Fix] [KV1] Skip route (and line) if no vj above (instead of fail pro…
ArnaudOggy Oct 11, 2019
81e7ecd
Merge pull request #410 from ArnaudOggy/fix_kv1_orphan_routes
mergify[bot] Oct 14, 2019
72a88ac
Force a boolean to be 0 or 1 in deserialization
woshilapin Oct 14, 2019
4e70b2c
Merge branch 'master' into upgrade-minidom
mergify[bot] Oct 14, 2019
26858fb
Merge pull request #409 from datanel/upgrade-minidom
mergify[bot] Oct 14, 2019
f085744
Remove useless packages
Oct 11, 2019
cca8c24
Merge branch 'master' into only-child
mergify[bot] Oct 14, 2019
940f844
Merge pull request #408 from woshilapin/only-child
mergify[bot] Oct 14, 2019
83c445d
Merge pull request #1 from woshilapin/update_gtfs_parser-from
patochectp Oct 14, 2019
7151602
Merge pull request #2 from woshilapin/update_gtfs_parser-tests
patochectp Oct 14, 2019
d8503f0
Merge branch 'master' into remove_packages
mergify[bot] Oct 14, 2019
f047bd1
Merge pull request #411 from datanel/remove_packages
mergify[bot] Oct 14, 2019
3ccc041
read / write pathways
patochectp Sep 12, 2019
d482f64
read / write levels.txt
patochectp Sep 13, 2019
2e6140c
manage level_id on stops
patochectp Sep 13, 2019
131b38c
update test
patochectp Sep 13, 2019
d64a748
stops.txt : read / write StopLocation
patochectp Sep 18, 2019
ac407c5
fix bug on id
patochectp Sep 19, 2019
0d8825b
refactoring
patochectp Sep 25, 2019
c5fe66e
update test
patochectp Sep 25, 2019
feaa4f3
gtfs parser refactoring
patochectp Oct 2, 2019
2ace6bc
ntfs parser refactoring
patochectp Oct 3, 2019
168f155
add test
patochectp Oct 7, 2019
dd7c3b4
consideration of comments
patochectp Oct 8, 2019
30fd259
fix fmt error
patochectp Oct 9, 2019
c9382c9
Transform Into implementation into From
woshilapin Oct 11, 2019
957f6be
Tests stops without geolocation and pathways with stairs
woshilapin Oct 11, 2019
c2c788c
fix bug after rebase
patochectp Oct 15, 2019
90e1b13
Merge pull request #3 from woshilapin/update_gtfs_parser-bool
patochectp Oct 15, 2019
3f64e4b
Force a boolean to be 0 or 1 in deserialization
patochectp Oct 15, 2019
d8de89a
add conditional compilation : stop_location
patochectp Oct 15, 2019
8d9037a
update fixture with new version ntfs
patochectp Oct 15, 2019
b60f26f
Manage pathways refactoring
ArnaudOggy Oct 15, 2019
bf086d9
Merge pull request #4 from ArnaudOggy/update_gtfs_parser
patochectp Oct 15, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/add_prefix.rs
Original file line number Diff line number Diff line change
@@ -76,6 +76,7 @@ impl AddPrefix for Collections {
self.ticket_uses.add_prefix(&prefix);
self.ticket_use_perimeters.add_prefix(&prefix);
self.ticket_use_restrictions.add_prefix(&prefix);
self.pathways.add_prefix(&prefix);
}
}

3 changes: 2 additions & 1 deletion src/gtfs/mod.rs
Original file line number Diff line number Diff line change
@@ -271,7 +271,7 @@ where
collections.comments = comments;
read::manage_stop_times(&mut collections, file_handler)?;
read::manage_frequencies(&mut collections, file_handler)?;

collections.pathways = read_utils::read_opt_collection(file_handler, "pathways.txt")?;
collections.sanitize()?;

//add prefixes
@@ -423,6 +423,7 @@ pub fn write<P: AsRef<Path>>(model: Model, path: P) -> Result<()> {
&model.stop_time_headsigns,
)?;
write::write_shapes(path, &model.geometries)?;
write_collection_with_id(path, "pathways.txt", &model.pathways)?;

Ok(())
}
1 change: 0 additions & 1 deletion src/gtfs/read.rs
Original file line number Diff line number Diff line change
@@ -448,7 +448,6 @@ pub fn read_agency<H>(
where
for<'a> &'a mut H: FileHandler,
{
info!("Reading agency.txt");
let filename = "agency.txt";
let gtfs_agencies = read_objects::<_, Agency>(file_handler, filename)?;
let networks = gtfs_agencies
1 change: 1 addition & 0 deletions src/model.rs
Original file line number Diff line number Diff line change
@@ -68,6 +68,7 @@ pub struct Collections {
pub ticket_prices: Collection<TicketPrice>,
pub ticket_use_perimeters: Collection<TicketUsePerimeter>,
pub ticket_use_restrictions: Collection<TicketUseRestriction>,
pub pathways: CollectionWithId<Pathway>,
}

impl Collections {
122 changes: 64 additions & 58 deletions src/ntfs/mod.rs
Original file line number Diff line number Diff line change
@@ -22,12 +22,14 @@ pub mod filter;
mod read;
mod write;

use crate::common_format;
use crate::model::{Collections, Model};
use crate::objects::*;
use crate::read_utils;
use crate::utils::*;
use crate::Result;
use crate::{
common_format::{manage_calendars, write_calendar_dates},
model::{Collections, Model},
objects::*,
read_utils,
utils::*,
Result,
};
use chrono::NaiveDateTime;
use derivative::Derivative;
use log::info;
@@ -171,7 +173,8 @@ pub fn read<P: AsRef<path::Path>>(path: P) -> Result<Model> {
collections.ticket_prices = make_opt_collection(path, "ticket_prices.txt")?;
collections.ticket_use_perimeters = make_opt_collection(path, "ticket_use_perimeters.txt")?;
collections.ticket_use_restrictions = make_opt_collection(path, "ticket_use_restrictions.txt")?;
common_format::manage_calendars(&mut file_handle, &mut collections)?;
collections.pathways = make_opt_collection_with_id(path, "pathways.txt")?;
manage_calendars(&mut file_handle, &mut collections)?;
read::manage_geometries(&mut collections, path)?;
read::manage_feed_infos(&mut collections, path)?;
read::manage_stops(&mut collections, path)?;
@@ -199,28 +202,28 @@ pub fn write<P: AsRef<path::Path>>(
info!("Writing NTFS to {:?}", path);

write::write_feed_infos(path, &model.feed_infos, &model.datasets, current_datetime)?;
write::write_collection_with_id(path, "contributors.txt", &model.contributors)?;
write::write_collection_with_id(path, "datasets.txt", &model.datasets)?;
write::write_collection_with_id(path, "networks.txt", &model.networks)?;
write::write_collection_with_id(path, "commercial_modes.txt", &model.commercial_modes)?;
write::write_collection_with_id(path, "companies.txt", &model.companies)?;
write::write_collection_with_id(path, "lines.txt", &model.lines)?;
write::write_collection_with_id(path, "physical_modes.txt", &model.physical_modes)?;
write::write_collection_with_id(path, "equipments.txt", &model.equipments)?;
write::write_collection_with_id(path, "routes.txt", &model.routes)?;
write::write_collection_with_id(path, "trip_properties.txt", &model.trip_properties)?;
write::write_collection_with_id(path, "geometries.txt", &model.geometries)?;
write::write_collection(path, "transfers.txt", &model.transfers)?;
write::write_collection(path, "admin_stations.txt", &model.admin_stations)?;
write::write_collection_with_id(path, "tickets.txt", &model.tickets)?;
write::write_collection_with_id(path, "ticket_uses.txt", &model.ticket_uses)?;
write::write_collection(path, "ticket_prices.txt", &model.ticket_prices)?;
write::write_collection(
write_collection_with_id(path, "contributors.txt", &model.contributors)?;
write_collection_with_id(path, "datasets.txt", &model.datasets)?;
write_collection_with_id(path, "networks.txt", &model.networks)?;
write_collection_with_id(path, "commercial_modes.txt", &model.commercial_modes)?;
write_collection_with_id(path, "companies.txt", &model.companies)?;
write_collection_with_id(path, "lines.txt", &model.lines)?;
write_collection_with_id(path, "physical_modes.txt", &model.physical_modes)?;
write_collection_with_id(path, "equipments.txt", &model.equipments)?;
write_collection_with_id(path, "routes.txt", &model.routes)?;
write_collection_with_id(path, "trip_properties.txt", &model.trip_properties)?;
write_collection_with_id(path, "geometries.txt", &model.geometries)?;
write_collection(path, "transfers.txt", &model.transfers)?;
write_collection(path, "admin_stations.txt", &model.admin_stations)?;
write_collection_with_id(path, "tickets.txt", &model.tickets)?;
write_collection_with_id(path, "ticket_uses.txt", &model.ticket_uses)?;
write_collection(path, "ticket_prices.txt", &model.ticket_prices)?;
write_collection(
path,
"ticket_use_perimeters.txt",
&model.ticket_use_perimeters,
)?;
write::write_collection(
write_collection(
path,
"ticket_use_restrictions.txt",
&model.ticket_use_restrictions,
@@ -233,12 +236,13 @@ pub fn write<P: AsRef<path::Path>>(
&model.stop_time_ids,
)?;
write::write_collection(path, "frequencies.txt", &model.frequencies)?;
common_format::write_calendar_dates(path, &model.calendars)?;
write_calendar_dates(path, &model.calendars)?;
write::write_stops(path, &model.stop_points, &model.stop_areas)?;
write::write_comments(path, model)?;
write::write_codes(path, model)?;
write::write_object_properties(path, model)?;
write::write_fares_v1(path, &model)?;
write_collection_with_id(path, "pathways.txt", &model.pathways)?;

Ok(())
}
@@ -264,6 +268,9 @@ pub fn write_to_zip<P: AsRef<path::Path>>(
mod tests {
use super::*;
use crate::{read_utils::PathFileHandler, test_utils::*};
use super::Collections;
use super::{read, write};
use crate::common_format::{manage_calendars, write_calendar_dates, Availability};
use geo_types::line_string;
use pretty_assertions::assert_eq;
use serde;
@@ -280,7 +287,7 @@ mod tests {
{
let collection = CollectionWithId::new(objects).unwrap();
test_in_tmp_dir(|path| {
write::write_collection_with_id(path, "file.txt", &collection).unwrap();
write_collection_with_id(path, "file.txt", &collection).unwrap();
let des_collection = make_collection_with_id(path, "file.txt").unwrap();
assert_eq!(collection, des_collection);
});
@@ -293,7 +300,7 @@ mod tests {
{
let collection = Collection::new(objects);
test_in_tmp_dir(|path| {
write::write_collection(path, "file.txt", &collection).unwrap();
write_collection(path, "file.txt", &collection).unwrap();
let des_collection = make_opt_collection(path, "file.txt").unwrap();
assert_eq!(collection, des_collection);
});
@@ -698,16 +705,16 @@ mod tests {
fn equipments_serialization_deserialization() {
test_serialize_deserialize_collection_with_id(vec![Equipment {
id: "1".to_string(),
wheelchair_boarding: common_format::Availability::Available,
sheltered: common_format::Availability::InformationNotAvailable,
elevator: common_format::Availability::Available,
escalator: common_format::Availability::Available,
bike_accepted: common_format::Availability::Available,
bike_depot: common_format::Availability::Available,
visual_announcement: common_format::Availability::Available,
audible_announcement: common_format::Availability::Available,
appropriate_escort: common_format::Availability::Available,
appropriate_signage: common_format::Availability::Available,
wheelchair_boarding: Availability::Available,
sheltered: Availability::InformationNotAvailable,
elevator: Availability::Available,
escalator: Availability::Available,
bike_accepted: Availability::Available,
bike_depot: Availability::Available,
visual_announcement: Availability::Available,
audible_announcement: Availability::Available,
appropriate_escort: Availability::Available,
appropriate_signage: Availability::Available,
}]);
}

@@ -754,10 +761,10 @@ mod tests {

test_in_tmp_dir(|path| {
let mut handler = PathFileHandler::new(path.to_path_buf());
common_format::write_calendar_dates(path, &calendars).unwrap();
write_calendar_dates(path, &calendars).unwrap();

let mut collections = Collections::default();
common_format::manage_calendars(&mut handler, &mut collections).unwrap();
manage_calendars(&mut handler, &mut collections).unwrap();

assert_eq!(calendars, collections.calendars);
});
@@ -1031,16 +1038,15 @@ mod tests {
ser_collections.stop_time_comments = stop_time_comments;

test_in_tmp_dir(|path| {
write::write_collection_with_id(path, "lines.txt", &ser_collections.lines).unwrap();
write_collection_with_id(path, "lines.txt", &ser_collections.lines).unwrap();
write::write_stops(
path,
&ser_collections.stop_points,
&ser_collections.stop_areas,
)
.unwrap();
write::write_collection_with_id(path, "routes.txt", &ser_collections.routes).unwrap();
write::write_collection_with_id(path, "networks.txt", &ser_collections.networks)
.unwrap();
write_collection_with_id(path, "routes.txt", &ser_collections.routes).unwrap();
write_collection_with_id(path, "networks.txt", &ser_collections.networks).unwrap();
write::write_vehicle_journeys_and_stop_times(
path,
&ser_collections.vehicle_journeys,
@@ -1209,24 +1215,24 @@ mod tests {
test_serialize_deserialize_collection_with_id(vec![
TripProperty {
id: "1".to_string(),
wheelchair_accessible: common_format::Availability::Available,
bike_accepted: common_format::Availability::NotAvailable,
air_conditioned: common_format::Availability::InformationNotAvailable,
visual_announcement: common_format::Availability::Available,
audible_announcement: common_format::Availability::Available,
appropriate_escort: common_format::Availability::Available,
appropriate_signage: common_format::Availability::Available,
wheelchair_accessible: Availability::Available,
bike_accepted: Availability::NotAvailable,
air_conditioned: Availability::InformationNotAvailable,
visual_announcement: Availability::Available,
audible_announcement: Availability::Available,
appropriate_escort: Availability::Available,
appropriate_signage: Availability::Available,
school_vehicle_type: TransportType::Regular,
},
TripProperty {
id: "2".to_string(),
wheelchair_accessible: common_format::Availability::Available,
bike_accepted: common_format::Availability::NotAvailable,
air_conditioned: common_format::Availability::InformationNotAvailable,
visual_announcement: common_format::Availability::Available,
audible_announcement: common_format::Availability::Available,
appropriate_escort: common_format::Availability::Available,
appropriate_signage: common_format::Availability::Available,
wheelchair_accessible: Availability::Available,
bike_accepted: Availability::NotAvailable,
air_conditioned: Availability::InformationNotAvailable,
visual_announcement: Availability::Available,
audible_announcement: Availability::Available,
appropriate_escort: Availability::Available,
appropriate_signage: Availability::Available,
school_vehicle_type: TransportType::RegularAndSchool,
},
]);
1 change: 0 additions & 1 deletion src/ntfs/write.rs
Original file line number Diff line number Diff line change
@@ -24,7 +24,6 @@ use csv;
use failure::{bail, format_err, ResultExt};
use log::{info, warn};
use rust_decimal::{prelude::ToPrimitive, Decimal};
use serde;
use std::collections::{BTreeMap, BTreeSet, HashMap};
use std::convert::TryFrom;
use std::path;
50 changes: 50 additions & 0 deletions src/objects.rs
Original file line number Diff line number Diff line change
@@ -1057,6 +1057,56 @@ impl GetObjectType for StopPoint {
}
}

#[derivative(Default)]
#[derive(Derivative, Serialize, Deserialize, Debug, Clone, PartialEq, Eq, Hash)]
pub enum PathwayMode {
#[derivative(Default)]
#[serde(rename = "1")]
Walkway,
#[serde(rename = "2")]
Stairs,
#[serde(rename = "3")]
MovingSidewalk,
#[serde(rename = "4")]
Escalator,
#[serde(rename = "5")]
Elevator,
#[serde(rename = "6")]
FareGate,
#[serde(rename = "7")]
ExitGate,
}

#[derive(Serialize, Deserialize, Debug, PartialEq, Default, Clone)]
pub struct Pathway {
#[serde(rename = "pathway_id")]
pub id: String,
pub from_stop_id: String,
pub to_stop_id: String,
pub pathway_mode: PathwayMode,
#[serde(deserialize_with = "de_from_u8", serialize_with = "ser_from_bool")]
pub is_bidirectional: bool,
#[serde(default, deserialize_with = "de_option_positive_decimal")]
pub length: Option<Decimal>,
pub traversal_time: Option<u32>,
#[serde(default, deserialize_with = "de_option_non_null_integer")]
pub stair_count: Option<i16>,
pub max_slope: Option<f32>,
#[serde(default, deserialize_with = "de_option_positive_float")]
pub min_width: Option<f32>,
pub signposted_as: Option<String>,
pub reversed_signposted_as: Option<String>,
}

impl AddPrefix for Pathway {
fn add_prefix(&mut self, prefix: &str) {
self.id = prefix.to_string() + &self.id;
self.from_stop_id = prefix.to_string() + &self.from_stop_id;
self.to_stop_id = prefix.to_string() + &self.to_stop_id;
}
}
impl_id!(Pathway);

pub type Date = chrono::NaiveDate;

#[derive(Serialize, Deserialize, Debug, Clone, PartialEq)]
41 changes: 40 additions & 1 deletion src/read_utils.rs
Original file line number Diff line number Diff line change
@@ -196,14 +196,41 @@ where
O: for<'de> serde::Deserialize<'de>,
{
let (reader, path) = file_handler.get_file(file_name)?;

let file_name = path.file_name();
let basename = file_name.map_or(path.to_string_lossy(), |b| b.to_string_lossy());
info!("Reading {}", basename);
let mut rdr = csv::Reader::from_reader(reader);
Ok(rdr
.deserialize()
.collect::<StdResult<_, _>>()
.with_context(ctx_from_path!(path))?)
}

pub fn read_opt_objects<H, O>(file_handler: &mut H, file_name: &str) -> Result<Vec<O>>
where
for<'a> &'a mut H: FileHandler,
O: for<'de> serde::Deserialize<'de>,
{
let (reader, path) = file_handler.get_file_if_exists(file_name)?;
let file_name = path.file_name();
let basename = file_name.map_or(path.to_string_lossy(), |b| b.to_string_lossy());

match reader {
None => {
info!("Skipping {}", basename);
Ok(vec![])
}
Some(reader) => {
info!("Reading {}", basename);
let mut rdr = csv::Reader::from_reader(reader);
Ok(rdr
.deserialize()
.collect::<StdResult<_, _>>()
.with_context(ctx_from_path!(path))?)
}
}
}

/// Read a CollectionId from a zip in a file_handler
pub fn read_collection<H, O>(file_handler: &mut H, file_name: &str) -> Result<CollectionWithId<O>>
where
@@ -214,6 +241,18 @@ where
CollectionWithId::new(vec)
}

pub fn read_opt_collection<H, O>(
file_handler: &mut H,
file_name: &str,
) -> Result<CollectionWithId<O>>
where
for<'a> &'a mut H: FileHandler,
O: for<'de> serde::Deserialize<'de> + Id<O>,
{
let vec = read_opt_objects(file_handler, file_name)?;
CollectionWithId::new(vec)
}

#[cfg(test)]
mod tests {
use super::*;
104 changes: 103 additions & 1 deletion src/utils.rs
Original file line number Diff line number Diff line change
@@ -186,9 +186,66 @@ where
Ok(number)
} else {
Err(D::Error::invalid_value(
Other("strictly negative integer number"),
&"positive integer number",
))
}
}

pub fn de_option_positive_decimal<'de, D>(deserializer: D) -> Result<Option<Decimal>, D::Error>
where
D: serde::Deserializer<'de>,
{
use serde::{
de::{Error, Unexpected::Other},
Deserialize,
};
let option = <Option<Decimal> as Deserialize<'de>>::deserialize(deserializer)?;
match option {
Some(number) if number.is_sign_positive() => Ok(option),
None => Ok(None),
_ => Err(D::Error::invalid_value(
Other("strictly negative float number"),
&"positive float number",
))
)),
}
}

pub fn de_option_positive_float<'de, D>(deserializer: D) -> Result<Option<f32>, D::Error>
where
D: serde::Deserializer<'de>,
{
use serde::{
de::{Error, Unexpected::Other},
Deserialize,
};
let option = <Option<f32> as Deserialize<'de>>::deserialize(deserializer)?;
match option {
Some(number) if number.is_sign_positive() => Ok(option),
None => Ok(None),
_ => Err(D::Error::invalid_value(
Other("strictly negative float number"),
&"positive float number",
)),
}
}

pub fn de_option_non_null_integer<'de, D>(deserializer: D) -> Result<Option<i16>, D::Error>
where
D: serde::Deserializer<'de>,
{
use serde::{
de::{Error, Unexpected::Other},
Deserialize,
};
let option = <Option<i16> as Deserialize<'de>>::deserialize(deserializer)?;
match option {
Some(number) if number != 0 => Ok(option),
None => Ok(None),
_ => Err(D::Error::invalid_value(
Other("0"),
&"non null number",
)),
}
}

@@ -297,6 +354,51 @@ where
Ok(Collection::new(vec))
}

pub fn write_collection_with_id<T>(
path: &path::Path,
file: &str,
collection: &CollectionWithId<T>,
) -> crate::Result<()>
where
T: Id<T>,
T: serde::Serialize,
{
if collection.is_empty() {
return Ok(());
}
info!("Writing {}", file);
let path = path.join(file);
let mut wtr = csv::Writer::from_path(&path).with_context(ctx_from_path!(path))?;
for obj in collection.values() {
wtr.serialize(obj).with_context(ctx_from_path!(path))?;
}
wtr.flush().with_context(ctx_from_path!(path))?;

Ok(())
}

pub fn write_collection<T>(
path: &path::Path,
file: &str,
collection: &Collection<T>,
) -> crate::Result<()>
where
T: serde::Serialize,
{
if collection.is_empty() {
return Ok(());
}
info!("Writing {}", file);
let path = path.join(file);
let mut wtr = csv::Writer::from_path(&path).with_context(ctx_from_path!(path))?;
for obj in collection.values() {
wtr.serialize(obj).with_context(ctx_from_path!(path))?;
}
wtr.flush().with_context(ctx_from_path!(path))?;

Ok(())
}

macro_rules! skip_fail {
($res:expr) => {{
use log::warn;