'ml_dataframe' dependency updated, public API changed (#30)

Author: gyrdym

CHANGELOG.md

@@ -1,5 +1,12 @@
 # Changelog
 
+## 7.0.0
+- `ml_datframe` 1.0.0 supported
+- `featureNames` parameter renamed to `columnNames`
+- `featureIds` parameter renamed to `columnIndices`
+- `encodeAsIntegerLabels` renamed to `toIntegerLabels`
+- `encodeAsOneHotLabels` renamed to `toOneHotLabels`
+
 ## 6.0.1
 - `pubspec.yaml`: `ml_dataframe` dependency updated
 

README.md

@@ -27,14 +27,14 @@ Let's say, you have a dataset:
 ````
 
 Everything seems good for now. Say, you're about to train a classifier to predict if a person has diabetes. 
-But there is an obstacle - how can it possible to use the data in mathematical equations with string-value columns 
-(`Gender`, `Country`)? And things are getting even worse because of an empty (N/A) value in `Diabetes` column. There 
+But there is an obstacle - how can it be possible to use the data in mathematical equations with string-value columns 
+(`Gender`, `Country`)? And things are getting even worse because of an empty (N/A) value in the `Diabetes` column. There 
 should be a way to convert this data to a valid numerical representation. Here data preprocessing techniques come to play. 
 You should decide, how to convert string data (aka *categorical data*) to numbers and how to treat empty values. Of 
-course, you can come up with your own unique algorithms to do all of these operations, but, actually, there are a 
-bunch of well-known well-performed techniques for doing all the conversions.      
+course, you can come up with your unique algorithms to do all of these operations, but there are a lot of well-known 
+techniques for doing all the conversions.      
 
-The aim of the library - to give data scientists, who are interested in Dart programming language, these preprocessing 
+The aim of the library is to give data scientists, who are interested in Dart programming language, these preprocessing 
 techniques.
 
 ## Prerequisites
@@ -47,7 +47,7 @@ before doing preprocessing. An example with a part of pubspec.yaml:
 ````
 dependencies:
   ...
-  ml_dataframe: ^0.0.11
+  ml_dataframe: ^1.0.0
   ...
 ````
 
@@ -90,14 +90,14 @@ Why should we fit it? Categorical data encoder fitting - a process, when all the
 searched for in order to create an encoded labels list. After the fitting is complete, one may use the fitted encoder for 
 the new data of the same source. 
 
-In order to fit the encoder it's needed to create the entity and pass the fitting data as an argument to the 
+In order to fit the encoder, it's needed to create the entity and pass the fitting data as an argument to the 
 constructor, along with the features to be encoded:
 
 
 ````dart
 final encoder = Encoder.oneHot(
   dataFrame,
-  featureNames: featureNames,
+  columnNames: featureNames,
 );
 
 ````
@@ -108,56 +108,56 @@ Let's encode the features:
 final encoded = encoder.process(dataFrame);
 ````
 
-We used the same dataframe here - it's absolutely normal, since when we created the encoder, we just fit it with the 
+We used the same dataframe here - it's absolutely normal since when we created the encoder, we just fit it with the 
 dataframe, and now is the time to apply the dataframe to the fitted encoder.
 
-It's time to take a look at our processed data! Let's read it:
+It's time to take a look at our processed data. Let's read it:
 
 ````dart
 final data = encoded.toMatrix();
 
 print(data);
 ```` 
 
-In the output we will see just numerical data, that's exactly we wanted to reach.
+In the output we will see just numerical data, that's exactly what we wanted to reach.
 
 ### Label encoding
 
-Another one well-known encoding method. The technique is the same - first, we should fit the encoder and after that we
+Another well-known encoding method. The technique is the same - first, we should fit the encoder and after that, we
 may use this "trained" encoder in some applications:
 
 ````dart
 // fit encoder
 final encoder = Encoder.label(
   dataFrame,
-  featureNames: featureNames,
+  columnNames: featureNames,
 );
 
 // apply fitted encoder to data
 final encoded = encoder.process(dataFrame);
 ````
 
-### Numerical data normalizing
+### Numerical data normalization
 
-Sometimes we need to have our numerical features normalized, that means we need to treat every dataframe row as a 
+Sometimes we need to have our numerical features normalized, which means we need to treat every dataframe row as a 
 vector and divide this vector element-wise by its norm (Euclidean, Manhattan, etc.). To do so the library exposes
-`Normalizer` entity:
+`Normalizer` class:
 
 ````dart
 final normalizer = Normalizer(); // by default Euclidean norm will be used
 final transformed = normalizer.process(dataFrame);
 ```` 
 
-Please, notice, if your data has raw categorical values, the normalization will fail as it requires only numerical 
-values. In this case you should encode data (e.g. using one-hot encoding) before normalization.
+Please, notice, that if your data has raw categorical values, the normalization will fail as it requires only numerical 
+values. In this case, you should encode data (e.g. using one-hot encoding) before normalization.
 
 ### Data standardization
 
 A lot of machine learning algorithms require normally distributed data as their input. Normally distributed data 
-means that every dedicated to a feature column in the data has zero mean and unit variance. One may reach this
-requirement using `Standardizer` class. During creation of the entity all the columns mean values and deviation values
-are being extracted from the passed data and stored as fields of the class, in order to apply them to standardize the
-other (or the same that was used for creation of the Standardizer) data:
+means that every column in the data has zero mean and unit variance. One may reach this requirement using the 
+`Standardizer` class. During the creation of the class instance, all the columns' mean values and deviation values are 
+being extracted from the passed data and stored as fields of the class, in order to apply them to standardize the 
+other (or the same that was used for the creation of the Standardizer) data:
 
 ````dart
 final dataFrame = DataFrame([
@@ -175,7 +175,7 @@ final transformed = standardizer.process(dataFrame);
 
 ### Pipeline
 
-There is a convenient way to organize a bunch of data preprocessing operations - `Pipeline`:
+There is a convenient way to organize a sequence of data preprocessing operations - `Pipeline`:
 
 ````dart
 final pipeline = Pipeline(dataFrame, [
@@ -186,12 +186,12 @@ final pipeline = Pipeline(dataFrame, [
 ]);
 ````
 
-Once you create (or rather fit) a pipeline, you may use it farther in your application:
+Once you create (or rather fit) a pipeline, you may use it further in your application:
 
 ````dart
 final processed = pipeline.process(dataFrame);
 ````
 
 `encodeAsOneHotLabels`, `encodeAsIntegerLabels`, `normalize` and `standardize` are pipeable operator functions. 
-Pipeable operator function is a factory, that takes fitting data and creates a fitted pipeable entity (e.g., 
+The pipeable operator function is a factory that takes fitting data and creates a fitted pipeable entity (e.g., 
 `Normalizer` instance)  

benchmark/main.dart

@@ -0,0 +1 @@
+

example/black_friday/black_friday.dart

@@ -2,26 +2,35 @@ import 'package:ml_dataframe/ml_dataframe.dart';
 import 'package:ml_preprocessing/ml_preprocessing.dart';
 
 Future processDataSetWithCategoricalData() async {
-  final dataFrame = await fromCsv('example/black_friday/black_friday.csv',
-    columnNames: ['Gender', 'Age', 'City_Category',
-      'Stay_In_Current_City_Years', 'Marital_Status'],
+  final dataFrame = await fromCsv(
+    'example/black_friday/black_friday.csv',
+    columnNames: [
+      'Gender',
+      'Age',
+      'City_Category',
+      'Stay_In_Current_City_Years',
+      'Marital_Status'
+    ],
   );
 
   final encoded = Encoder.oneHot(
     dataFrame,
-    featureNames: ['Gender', 'Age', 'City_Category',
-      'Stay_In_Current_City_Years', 'Marital_Status'],
+    columnNames: [
+      'Gender',
+      'Age',
+      'City_Category',
+      'Stay_In_Current_City_Years',
+      'Marital_Status'
+    ],
   ).process(dataFrame);
 
   final observations = encoded.toMatrix();
   final genderEncoded = observations.sample(columnIndices: [0, 1]);
   final ageEncoded = observations.sample(columnIndices: [2, 3, 4, 5, 6, 7, 8]);
-  final cityCategoryEncoded = observations
-      .sample(columnIndices: [9, 10, 11]);
-  final stayInCityEncoded = observations
-      .sample(columnIndices: [12, 13, 14, 15, 16]);
-  final maritalStatusEncoded = observations
-      .sample(columnIndices: [17, 18]);
+  final cityCategoryEncoded = observations.sample(columnIndices: [9, 10, 11]);
+  final stayInCityEncoded =
+      observations.sample(columnIndices: [12, 13, 14, 15, 16]);
+  final maritalStatusEncoded = observations.sample(columnIndices: [17, 18]);
 
   print('Features:');
 

example/main.dart

@@ -1,20 +1,16 @@
 import 'package:ml_dataframe/ml_dataframe.dart';
 import 'package:ml_preprocessing/ml_preprocessing.dart';
-import 'package:ml_preprocessing/src/encoder/encode_as_integer_labels.dart';
-import 'package:ml_preprocessing/src/encoder/encode_as_one_hot_labels.dart';
-import 'package:ml_preprocessing/src/pipeline/pipeline.dart';
 
 Future main() async {
-  final dataFrame = await fromCsv('example/dataset.csv',
-      columns: [0, 1, 2, 3]);
+  final dataFrame = await fromCsv('example/dataset.csv', columns: [0, 1, 2, 3]);
 
   final pipeline = Pipeline(dataFrame, [
-    encodeAsOneHotLabels(
-      featureNames: ['position'],
+    toOneHotLabels(
+      columnNames: ['position'],
       headerPostfix: '_position',
     ),
-    encodeAsIntegerLabels(
-      featureNames: ['country'],
+    toIntegerLabels(
+      columnNames: ['country'],
     ),
   ]);
 

lib/ml_preprocessing.dart

@@ -1,7 +1,7 @@
 export 'package:ml_linalg/norm.dart';
-export 'package:ml_preprocessing/src/encoder/encode_as_integer_labels.dart';
-export 'package:ml_preprocessing/src/encoder/encode_as_one_hot_labels.dart';
 export 'package:ml_preprocessing/src/encoder/encoder.dart';
+export 'package:ml_preprocessing/src/encoder/to_integer_labels.dart';
+export 'package:ml_preprocessing/src/encoder/to_one_hot_labels.dart';
 export 'package:ml_preprocessing/src/encoder/unknown_value_handling_type.dart';
 export 'package:ml_preprocessing/src/normalizer/normalize.dart';
 export 'package:ml_preprocessing/src/normalizer/normalizer.dart';

lib/src/encoder/encode_as_integer_labels.dart

@@ -7,31 +7,35 @@ import 'package:ml_preprocessing/src/pipeline/pipeable.dart';
 
 /// Categorical data encoder factory
 abstract class Encoder implements Pipeable {
-  factory Encoder.oneHot(DataFrame fittingData, {
-    Iterable<int>? featureIds,
-    Iterable<String>? featureNames,
+  factory Encoder.oneHot(
+    DataFrame fittingData, {
+    Iterable<int>? columnIndices,
+    Iterable<String>? columnNames,
     UnknownValueHandlingType unknownValueHandlingType =
         defaultUnknownValueHandlingType,
-  }) => EncoderImpl(
-    fittingData,
-    EncoderType.oneHot,
-    const SeriesEncoderFactoryImpl(),
-    featureNames: featureNames,
-    featureIds: featureIds,
-    unknownValueHandlingType: unknownValueHandlingType,
-  );
+  }) =>
+      EncoderImpl(
+        fittingData,
+        EncoderType.oneHot,
+        const SeriesEncoderFactoryImpl(),
+        columnNames: columnNames,
+        columnIndices: columnIndices,
+        unknownValueHandlingType: unknownValueHandlingType,
+      );
 
-  factory Encoder.label(DataFrame fittingData, {
-    Iterable<int>? featureIds,
-    Iterable<String>? featureNames,
+  factory Encoder.label(
+    DataFrame fittingData, {
+    Iterable<int>? columnIndices,
+    Iterable<String>? columnNames,
     UnknownValueHandlingType unknownValueHandlingType =
         defaultUnknownValueHandlingType,
-  }) => EncoderImpl(
-    fittingData,
-    EncoderType.label,
-    const SeriesEncoderFactoryImpl(),
-    featureNames: featureNames,
-    featureIds: featureIds,
-    unknownValueHandlingType: unknownValueHandlingType,
-  );
+  }) =>
+      EncoderImpl(
+        fittingData,
+        EncoderType.label,
+        const SeriesEncoderFactoryImpl(),
+        columnNames: columnNames,
+        columnIndices: columnIndices,
+        unknownValueHandlingType: unknownValueHandlingType,
+      );
 }