init with video preprocess

Author: Augists

.gitignore

@@ -0,0 +1,6 @@
+.gitignore
+archived
+datasets
+best_models
+outputs
+save_models

.idea/.name

@@ -0,0 +1,55 @@
+# GaitFi: Robust Device-Free Human Identification via WiFi and Vision Multimodal Learning [[link]](https://doi.org/10.48550/arXiv.2208.14326)
+
+## Introduction
+As an important biomarker for human identification, human gait can be collected at a distance by passive sensors without subject cooperation, which plays an essential role in crime prevention, security detection and other human identification applications. At present, most research works are based on cameras and computer vision techniques to perform gait recognition. However, vision-based methods are not reliable when confronting poor illuminations, leading to degrading performances. In this paper, we propose a novel multimodal gait recognition method, namely GaitFi, which leverages WiFi signals and videos for human identification. In GaitFi, Channel State Information (CSI) that reflects the multi-path propagation of WiFi is collected to capture human gaits, while videos are captured by cameras. To learn robust gait information, we propose a Lightweight Residual Convolution Network (LRCN) as the backbone network, and further propose the two-stream GaitFi by integrating WiFi and vision features for the gait retrieval task. The GaitFi is trained by the triplet loss and classification loss on different levels of features. Extensive experiments are conducted in the real world, which demonstrates that the GaitFi outperforms state-of-the-art gait recognition methods based on single WiFi or camera, achieving 94.2\% for human identification tasks of 12 subjects.
+
+![framework](Framework.jpg)
+
+## Requirements
+
+```
+scipy - 1.5.4
+numpy - 1.21.5
+torchvision - 0.11.2
+pytorch - 1.7.0
+```
+
+
+
+## Training
+Train using vision modality only: `python train.py --input_type image`
+
+Train using WiFi modality only: `python train.py --input_type mat`
+
+Train using both vision and WiFi modality: `python train.py --input_type both`
+
+## Testing
+Copy the model saved to save_models in training to best_models
+
+Test using vision modality only: `python test.py --input_type image`
+
+Test using WiFi modality only: `python test.py --input_type mat`
+
+Test using WiFi modality only: `python test.py --input_type both`
+
+
+
+## Model
+
+The GaitFi has the following components:
+
+- ***class*** **CNN** : LRCN block
+- ***class*** **RNN** : LSTM block
+- ***class*** **CRNN**: Fusion mechanism of WiFi and vision modalities
+
+## Reference
+
+```
+@ARTICLE{9887951,
+  author={Deng, Lang and Yang, Jianfei and Yuan, Shenghai and Zou, Han and Lu, Chris Xiaoxuan and Xie, Lihua},
+  journal={IEEE Internet of Things Journal}, 
+  title={GaitFi: Robust Device-Free Human Identification via WiFi and Vision Multimodal Learning}, 
+  year={2022},
+  publisher={IEEE},
+  doi={10.1109/JIOT.2022.3203559}}
+```

.idea/GaitFi_model.iml

@@ -0,0 +1,109 @@
+"""
+分割数据集
+训练集和测试集比例为 8:2
+"""
+
+import os
+import shutil
+from pathlib import Path
+
+import numpy as np
+import pandas as pd
+
+
+def check_create_path(path):
+    """
+    check and mkdir if not exist
+    recursive
+    """
+    if not os.path.exists(path):
+        os.makedirs(path)
+
+
+def check_empty_dirs(path):
+    """
+    check if the directory is empty
+    recursive
+    :return: True if empty
+    """
+    for root, dirs, files in os.walk(path, topdown=False):
+        if files:
+            return False
+        else:
+            if dirs:
+                for dir in dirs:
+                    if not check_empty_dirs(os.path.join(root, dir)):
+                        return False
+            os.rmdir(root)
+    return True
+
+
+# working: str = f'occlusion/board'
+# working: str = f'occlusion/cabinet'
+# working: str = f'occlusion/desk'
+working: str = f'light/dark'
+
+
+FILE = Path(__file__).resolve()
+ROOT = FILE.parents[0]  # current directory
+
+DATASETS = ROOT / 'datasets'
+RAW = DATASETS / 'raw'  # ./datasets/raw
+WORKING = RAW / working  # ./datasets/raw/occlusion/board
+OUTPUT = DATASETS / working  # ./datasets/occlusion/board
+check_create_path(OUTPUT)
+
+
+def split(path):
+    """
+    对每个模态、每个动作分别划分数据集和测试集
+    """
+    for mod in os.listdir(path):
+        MOD = path / mod
+
+        for label in os.listdir(MOD):
+            LABEL = MOD / label  # ./datasets/raw/occlusion/board/CSI/clap
+
+            train, test = split_train_test(LABEL)  # pandas.DataFrame
+
+            TRAIN = OUTPUT / 'train' / mod / label  # ./datasets/occlusion/board/train/CSI/clap
+            TEST = OUTPUT / 'test' / mod / label  # ./datasets/occlusion/board/test/CSI/clap
+            check_create_path(TRAIN)
+            check_create_path(TEST)
+
+            move_datasets(train, TRAIN)
+            move_datasets(test, TEST)
+
+
+def move_datasets(files, path):
+    """
+    :param files: pandas.DataFrame file list
+    :param path: output path
+    :return: None
+    """
+    # print(files)
+    # print(files.dtypes)
+    # print(files[0])
+    for file in files[0]:
+        # print(file)
+        shutil.move(file, path)
+
+
+def split_train_test(path, train_radio=0.8):
+    FILES = os.listdir(path)
+    FILES = [path / file for file in FILES]
+    FILES = pd.DataFrame(FILES)
+
+    np.random.seed(42)
+    shuffled_indices = np.random.permutation(len(FILES))
+    train_set_size = int(len(FILES) * train_radio)
+    train_indices = shuffled_indices[:train_set_size]
+    test_indices = shuffled_indices[train_set_size:]
+
+    return FILES.iloc[train_indices], FILES.iloc[test_indices]
+
+
+if __name__ == '__main__':
+    split(WORKING)
+    if not check_empty_dirs(WORKING):
+        print("Some files still in raw directory, check manually")