Skip to content

Commit

Permalink
Added an option to override the culture for the CSV data source
Browse files Browse the repository at this point in the history
  • Loading branch information
YuriyIvon committed Jul 8, 2023
1 parent d52652e commit cfbe0ab
Show file tree
Hide file tree
Showing 3 changed files with 11 additions and 2 deletions.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

# Database Benchmark
![Application Status](https://github.com/YuriyIvon/DatabaseBenchmark/actions/workflows/build.yml/badge.svg)

Expand Down Expand Up @@ -96,7 +97,7 @@ DatabaseBenchmark import --DatabaseType=Postgres --ConnectionString="Host=localh

Here file [SalesSqlServerDataSource.json](https://github.com/YuriyIvon/DatabaseBenchmark/blob/main/samples/Sales/SalesSqlServerDataSource.json) defines where and how to get source data. With relational databases, a raw query stored in a file referenced by `QueryFilePath` parameter must return all columns declared in the target table definition. Extra columns are ignored. For those databases where the container name doesn't appear in a query, there is an optional data source parameter `TableName`.

CSV data source has an optional parameter `DataSource.Csv.Delimiter`, which allows you to override the default column delimiter.
CSV data source has an optional parameter `DataSource.Csv.Delimiter`, which allows you to override the default column delimiter, and `DataSource.Csv.Culture` to override the default system culture used for parsing the input CSV file.

If column names in the data source don't match table columns (for example, when CSV headers contain space characters), a mapping can be applied by specifying `MappingFilePath` parameter pointing to a JSON file with column mappings. An object in this file must have `Columns` array where each item consists of two fields - `SourceColumnName` and `TableColumnName`.

Expand Down
7 changes: 6 additions & 1 deletion src/DatabaseBenchmark/DataSources/Csv/CsvDataSource.cs
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,12 @@ public bool Read()
private void Open()
{
var options = _optionsProvider.GetOptions<CsvDataSourceOptions>();
var configuration = new CsvConfiguration(CultureInfo.InvariantCulture);

var culture = options.Culture != null
? CultureInfo.GetCultureInfo(options.Culture)
: CultureInfo.CurrentCulture;

var configuration = new CsvConfiguration(culture);

if (options.Delimiter != null)
{
Expand Down
3 changes: 3 additions & 0 deletions src/DatabaseBenchmark/DataSources/Csv/CsvDataSourceOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,8 @@ public class CsvDataSourceOptions
{
[Option("Column delimiter")]
public string Delimiter { get; set; }

[Option("Culture identifier used for parsing (the current system culture is used by default)")]
public string Culture { get; set; }
}
}

0 comments on commit cfbe0ab

Please sign in to comment.