🔀 Merge pull request #21 from hucebot/dev

🔀 Dev

README.md

@@ -112,7 +112,11 @@ and unzip it, it should be following this structure:
 │           │   ├── ...
 
 ```
-Then run the script `dataset_preprocessing/teleopicubdataset_to_hdf5.py` to generate the dataset in the lib's format, with `--data_path` argument providing the path to the downloaded dataset, and the `--hdf5_path` argument giving the path and name of the generated hdf5 file (optional)
+Then run the script `dataset_preprocessing/teleopicubdataset_to_hdf5.py` to generate the dataset in the lib's format, with `--data_path` argument providing the path to the downloaded dataset, and the `--hdf5_path` argument giving the path and name of the generated hdf5 file (optional).  
+For example something like:
+```bash
+python dataset_preprocessing/teleopicubdataset_to_hdf5.py --data_path AndyData-lab-prescientTeleopICub/ --hdf5_path AndyData-lab-prescientTeleopICub.hdf5
+```
 
 
 #### H36MDataset
@@ -125,7 +129,11 @@ For [Human3.6M](http://vision.imar.ro/human3.6m/description.php) you need to dow
 |   ├── ...
 |   ├── S11
 ```
-Then run the script `dataset_preprocessing/h36mdataset_to_hdf5.py` to generate the dataset in the lib's format, with `--data_path` argument providing the path to the downloaded dataset, and the `--hdf5_path` argument giving the path and name of the generated hdf5 file (optional)
+Then run the script `dataset_preprocessing/h36mdataset_to_hdf5.py` to generate the dataset in the lib's format, with `--data_path` argument providing the path to the downloaded dataset, and the `--hdf5_path` argument giving the path and name of the generated hdf5 file (optional).  
+For example something like:
+```bash
+python dataset_preprocessing/h36mdataset_to_hdf5.py --data_path h36m/ --hdf5_path h36m.hdf5
+```
 
 
 #### AndyDataset
@@ -139,7 +147,11 @@ For [AndyDataset](https://andydataset.loria.fr/) you need to download the zip [h
 |       ├── ...
 |       ├── Participant_9875
 ```
-Then run the script `dataset_preprocessing/andydataset_to_hdf5.py` to generate the dataset in the lib's format, with `--data_path` argument providing the path to the downloaded dataset, and the `--hdf5_path` argument giving the path and name of the generated hdf5 file (optional)
+Then run the script `dataset_preprocessing/andydataset_to_hdf5.py` to generate the dataset in the lib's format, with `--data_path` argument providing the path to the downloaded dataset, and the `--hdf5_path` argument giving the path and name of the generated hdf5 file (optional).  
+For example something like:
+```bash
+python dataset_preprocessing/andydataset_to_hdf5.py --data_path AndyData-lab-onePerson/ --hdf5_path AndyData-lab-onePerson.hdf5
+```
 
 
 ## Predictors

benchmark/andydataset/train_simlpe_variations.py

@@ -36,7 +36,7 @@
     "model_config.name": [
         "siMLPe",
     ],
-    "model_config.loss_fn": [LossFunctions.MTRDLOSS, LossFunctions.MSELOSS],
+    "model_config.loss_fn": [LossFunctions.MTDLOSS, LossFunctions.MSELOSS],
     # "model_config.num_layers": [48],                 # commented out to leave default model values here instead
     # "model_config.spatial_fc_only": [True, False],   # commented out to leave default model values here instead
     # "model_config.dct": [True, False],               # commented out to leave default model values here instead

dataset_preprocessing/andydataset_to_hdf5.py

@@ -6,7 +6,7 @@
 
 
 DEFAULT_DATA_PATH = "data/datasets/AndyData-lab-onePerson/xens_mnvx"
-DEFAULT_GLOB_PATERN = "*.mvnx"
+DEFAULT_GLOB_PATTERN = "*.mvnx"
 DEFAULT_HDF5_PATH = "data/datasets/AndyData-lab-onePerson.hdf5"
 
 
@@ -16,9 +16,9 @@
         "--data_path", default=DEFAULT_DATA_PATH, help="path to the data directory"
     )
     parser.add_argument(
-        "--glob_patern",
-        default=DEFAULT_GLOB_PATERN,
-        help="pattern used to retreive the list of files to parse in the directory",
+        "--glob_pattern",
+        default=DEFAULT_GLOB_PATTERN,
+        help="pattern used to retrieve the list of files to parse in the directory",
     )
     parser.add_argument(
         "--hdf5_path", default=DEFAULT_HDF5_PATH, help="filepath to the created hdf5"
@@ -27,6 +27,6 @@
     args = parser.parse_args()
     hdf5_path = args.hdf5_path
     data_path = args.data_path
-    glob_patern = args.glob_patern
+    glob_pattern = args.glob_pattern
 
-    AndyDataset.create_hdf5(hdf5_path, data_path, glob_patern)
+    AndyDataset.create_hdf5(hdf5_path, data_path, glob_pattern)

dataset_preprocessing/h36mdataset_to_hdf5.py

@@ -6,7 +6,7 @@
 
 
 DEFAULT_DATA_PATH = "data/datasets/h36m"
-DEFAULT_GLOB_PATERN = "*.txt"
+DEFAULT_GLOB_PATTERN = "*.txt"
 DEFAULT_HDF5_PATH = "data/datasets/h36m.hdf5"
 
 
@@ -16,9 +16,9 @@
         "--data_path", default=DEFAULT_DATA_PATH, help="path to the data directory"
     )
     parser.add_argument(
-        "--glob_patern",
-        default=DEFAULT_GLOB_PATERN,
-        help="pattern used to retreive the list of files to parse in the directory",
+        "--glob_pattern",
+        default=DEFAULT_GLOB_PATTERN,
+        help="pattern used to retrieve the list of files to parse in the directory",
     )
     parser.add_argument(
         "--hdf5_path", default=DEFAULT_HDF5_PATH, help="filepath to the created hdf5"
@@ -27,6 +27,6 @@
     args = parser.parse_args()
     hdf5_path = args.hdf5_path
     data_path = args.data_path
-    glob_patern = args.glob_patern
+    glob_pattern = args.glob_pattern
 
-    H36MDataset.create_hdf5(hdf5_path, data_path, glob_patern)
+    H36MDataset.create_hdf5(hdf5_path, data_path, glob_pattern)

dataset_preprocessing/teleopicubdataset_to_hdf5.py

@@ -6,7 +6,7 @@
 
 
 DEFAULT_DATA_PATH = "data/datasets/AndyData-lab-prescientTeleopICub"
-DEFAULT_GLOB_PATERN = "*.csv"
+DEFAULT_GLOB_PATTERN = "*.csv"
 DEFAULT_HDF5_PATH = "data/datasets/AndyData-lab-prescientTeleopICub.hdf5"
 
 
@@ -16,9 +16,9 @@
         "--data_path", default=DEFAULT_DATA_PATH, help="path to the data directory"
     )
     parser.add_argument(
-        "--glob_patern",
-        default=DEFAULT_GLOB_PATERN,
-        help="pattern used to retreive the list of files to parse in the directory",
+        "--glob_pattern",
+        default=DEFAULT_GLOB_PATTERN,
+        help="pattern used to retrieve the list of files to parse in the directory",
     )
     parser.add_argument(
         "--hdf5_path", default=DEFAULT_HDF5_PATH, help="filepath to the created hdf5"
@@ -27,6 +27,6 @@
     args = parser.parse_args()
     hdf5_path = args.hdf5_path
     data_path = args.data_path
-    glob_patern = args.glob_patern
+    glob_pattern = args.glob_pattern
 
-    TeleopIcubDataset.create_hdf5(hdf5_path, data_path, glob_patern)
+    TeleopIcubDataset.create_hdf5(hdf5_path, data_path, glob_pattern)

docs/configuration_files.rst

@@ -1,6 +1,6 @@
-########################
-API Configurations Files
-########################
+######################
+Configurations Classes
+######################
 
 .. toctree::
    :maxdepth: 1
@@ -14,7 +14,7 @@ Dataset
 =============================
 .. autoclass:: prescyent.dataset.config.TrajectoriesDatasetConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields, check_context_keys
+   :exclude-members: model_config, model_extra, model_fields, check_context_keys, model_computed_fields, unserialize_features, generate_random_seed_if_none
    :undoc-members:
    :inherited-members:
    :show-inheritance:
@@ -26,7 +26,7 @@ Dataset
 
 .. autoclass:: prescyent.dataset.datasets.human36m.config.H36MDatasetConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields, check_context_keys
+   :exclude-members: model_config, model_extra, model_fields, check_context_keys, model_computed_fields, unserialize_features, generate_random_seed_if_none
    :undoc-members:
    :inherited-members:
    :show-inheritance:
@@ -38,7 +38,7 @@ Dataset
 
 .. autoclass:: prescyent.dataset.datasets.andydataset.config.AndyDatasetConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields, check_context_keys
+   :exclude-members: model_config, model_extra, model_fields, check_context_keys, model_computed_fields, unserialize_features, generate_random_seed_if_none
    :undoc-members:
    :inherited-members:
    :show-inheritance:
@@ -50,7 +50,7 @@ Dataset
 
 .. autoclass:: prescyent.dataset.datasets.teleop_icub.config.TeleopIcubDatasetConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields, check_context_keys
+   :exclude-members: model_config, model_extra, model_fields, check_context_keys, model_computed_fields, unserialize_features, generate_random_seed_if_none
    :undoc-members:
    :inherited-members:
    :show-inheritance:
@@ -62,7 +62,7 @@ Dataset
 
 .. autoclass:: prescyent.dataset.datasets.synthetic_circle_clusters.config.SCCDatasetConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields, check_context_keys
+   :exclude-members: model_config, model_extra, model_fields, check_context_keys, model_computed_fields, unserialize_features, check_list_size_matches, generate_random_seed_if_none
    :undoc-members:
    :inherited-members:
    :show-inheritance:
@@ -74,22 +74,7 @@ Dataset
 
 .. autoclass:: prescyent.dataset.datasets.synthetic_simple_trajs.config.SSTDatasetConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields, check_context_keys
-   :undoc-members:
-   :inherited-members:
-   :show-inheritance:
-
-******
-Scaler
-******
-
-
-**ScalerConfig**
-================
-
-.. autoclass:: prescyent.scaler.config.ScalerConfig
-   :members:
-   :exclude-members: model_config, model_extra, model_fields
+   :exclude-members: model_config, model_extra, model_fields, check_context_keys, model_computed_fields, unserialize_features, generate_random_seed_if_none
    :undoc-members:
    :inherited-members:
    :show-inheritance:
@@ -104,7 +89,7 @@ Predictor
 
 .. autoclass:: prescyent.predictor.config.PredictorConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields
+   :exclude-members: model_config, model_extra, model_fields, model_computed_fields, unserialize_features, generate_random_seed_if_none
    :undoc-members:
    :inherited-members:
    :show-inheritance:
@@ -116,11 +101,11 @@ Predictor
 
 .. autoclass:: prescyent.predictor.promp.config.PrompConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields
+   :exclude-members: model_config, model_extra, model_fields, model_computed_fields, unserialize_features, generate_random_seed_if_none
    :undoc-members:
    :inherited-members:
    :show-inheritance:
-   
+
 ----
 
 
@@ -129,7 +114,7 @@ Predictor
 
 .. autoclass:: prescyent.predictor.lightning.configs.module_config.ModuleConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields
+   :exclude-members: model_config, model_extra, model_fields, model_computed_fields, unserialize_features, generate_random_seed_if_none
    :undoc-members:
    :inherited-members:
    :show-inheritance:
@@ -142,7 +127,7 @@ Predictor
 
 .. autoclass:: prescyent.predictor.lightning.models.sequence.seq2seq.config.Seq2SeqConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields
+   :exclude-members: model_config, model_extra, model_fields, model_computed_fields, unserialize_features, generate_random_seed_if_none
    :undoc-members:
    :inherited-members:
    :show-inheritance:
@@ -154,7 +139,7 @@ Predictor
 
 .. autoclass:: prescyent.predictor.lightning.models.sequence.simlpe.config.SiMLPeConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields
+   :exclude-members: model_config, model_extra, model_fields, model_computed_fields, unserialize_features, generate_random_seed_if_none
    :undoc-members:
    :inherited-members:
    :show-inheritance:
@@ -166,7 +151,7 @@ Predictor
 
 .. autoclass:: prescyent.predictor.lightning.models.sequence.mlp.config.MlpConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields
+   :exclude-members: model_config, model_extra, model_fields, model_computed_fields, unserialize_features, generate_random_seed_if_none, check_context_attributes
    :undoc-members:
    :inherited-members:
    :show-inheritance:
@@ -178,7 +163,7 @@ Predictor
 
 .. autoclass:: prescyent.predictor.lightning.models.autoreg.sarlstm.config.SARLSTMConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields, name_sup_or_equal_one
+   :exclude-members: model_config, model_extra, model_fields, name_sup_or_equal_one, model_computed_fields, unserialize_features, generate_random_seed_if_none
    :undoc-members:
    :inherited-members:
    :show-inheritance:
@@ -188,24 +173,28 @@ Training
 ********
 
 
-**OptimizerConfig**
-===================
+**TrainingConfig**
+==================
 
-.. autoclass:: prescyent.predictor.lightning.configs.optimizer_config.OptimizerConfig
+.. autoclass:: prescyent.predictor.lightning.configs.training_config.TrainingConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields
+   :exclude-members: model_config, model_extra, model_fields, model_computed_fields, unserialize_features, generate_random_seed_if_none, training_has_at_least_one_positive_limit
    :undoc-members:
    :inherited-members:
    :show-inheritance:
 
-----
 
-**TrainingConfig**
-==================
+******
+Scaler
+******
 
-.. autoclass:: prescyent.predictor.lightning.configs.training_config.TrainingConfig
+
+**ScalerConfig**
+================
+
+.. autoclass:: prescyent.scaler.config.ScalerConfig
    :members:
-   :exclude-members: model_config, model_extra, model_fields
+   :exclude-members: model_config, model_extra, model_fields, model_computed_fields, unserialize_features, generate_random_seed_if_none
    :undoc-members:
    :inherited-members:
-   :show-inheritance:
+   :show-inheritance:

docs/datasets.md

@@ -6,10 +6,12 @@ Import all of theses datasets and their configs from `prescyent.dataset`.
 
 ## Downloadable Datasets
 
+Preprocess the dataset downloaded from source to turn it into the hdf5_file that prescyent is reading.
+
 ### AndyDataset
 
 This in-house dataset includes measurements of human movements and forces during the execution of different manual tasks.  
-For the HDF5 dataset we only extracted the information from xsens' .mvnx files  
+For the HDF5 dataset we only extracted the information from xsens' .mvnx files.  
 Further descriptions, original data and additional data for this dataset can be found [here](https://zenodo.org/records/3254403#.Y_9fwBeZMVk).  
 
 This hdf5 dataset contains the following features from the xsens motion capture, at each time frame at 240Hz:
@@ -30,12 +32,11 @@ This hdf5 dataset contains the following features from the xsens motion capture,
 Subset of the [h36m dataset](http://vision.imar.ro/human3.6m/description.php) used in [numerous benchmarks](https://paperswithcode.com/dataset/human3-6m), such as in Human Pose Forecasting.  
 This subsets contains only the exponential maps for all joints of the actors.  
 It was originally downloaded from a [stanford backup](http://www.cs.stanford.edu/people/ashesh/h3.6m.zip) (down now)  
+Uploaded [here](https://drive.google.com/file/d/15OAOUrva1S-C_BV8UgPORcwmWG2ul4Rk/view) by the SiMLPe's team.  
 
 The HDF5 file includes all of the trajectories into one single file, following the concepts used in the prescyent library.  
 For each traj, we have the coordinates and rotations for the 32 joints, inferred from the expmap and skeleton infos at 50Hz, as in [siMLPe's implementation](https://github.com/dulucas/siMLPe)  
 
-To use it in the [PreScyent library](https://github.com/hucebot/prescyent/), simply download the .hdf5 file and place it in your data folder (described in your DatasetConfig, default is "data/datasets/")  
-
 [Config](configuration_files.rst#h36mdatasetconfig)
 
 ### TeleopIcubDataset
@@ -58,6 +59,7 @@ Some datasets are created by the library and do not require to download a file.
 ### SCCDataset
 
 Synthetic Dataset generating circular 2D trajectories with more or less noise given the configuration parameters.  
+Given starting (x, y) pairs, and variations to the circle radius, and amount of noise in the circle, we generate trajectories with different starting points and slight variations, that can be rendered as a set of 2D circular trajectories.  
 
 [Config](configuration_files.rst#sccdatasetconfig)