Skip to content

Latest commit

 

History

History
256 lines (222 loc) · 13.6 KB

README_EN.md

File metadata and controls

256 lines (222 loc) · 13.6 KB
Raw

[Japanese/English]

Tensorflow2-ObjectDetectionAPI-Colab-Hands-On

mkv4t-6ilnu

Hands-on documentation for the Tensorflow2 Object Detection API.
Annotation with VoTT is performed on the local PC, and learning-inference is performed on Colaboratory.

This repository contains the following:

  • Dataset for learning (Annotation not implemented)
  • Test dataset
  • Model for fine-tuning(EffientDet D0)
  • Script for Google Colaboratory(Environment setting, model training, inference result confirmation)
Directory structure 
│ [Colaboratory]Tensorflow2_ObjectDetectionAPI_Colab_Hands_On.ipynb
|
├─01_train_data─┬─000000.jpg
│               │     :
│               └─000049.jpg
│      
├─02_tfrecord
│      
├─03_pretrained_model─efficientdet_d0_coco17_tpu-32─┬─pipeline.config
│                                                   ├─checkpoint──┬─checkpoint
│                                                   │             ├─ckpt-0.data-00000-of-00001
│                                                   │             └─ckpt-0.index
│                                                   └─saved_model─┬─saved_model.pb
│                                                                 └─variables─┬─variables.data-00000-of-00001
│                                                                             └─variables.index
│
└─04_test_data─┬─000050.jpg
               │     :
               └─000099.jpg

[Colaboratory]Tensorflow2_ObjectDetectionAPI_Colab_Hands_On.ipynb

Script for Google Colaboratory(Environment setting, model training, inference result confirmation)

01_train_data

Dataset for learning (Annotation not implemented)

02_tfrecord

Annotated TFRecord storage location

03_pretrained_mode

Model for fine-tuning(EffientDet D0)

04_test_data

Test dataset

Requirement

Tensorflow 2.3.0

Overview

This hands-on assumes about 2 hours.

  1. VoTT：Annotation(30-60minutes)
  2. Colaboratory：Object Detection API Setting
  3. Pipeline-config correction
  4. Colaboratory：Model training(About 25minutes)
  5. Colaboratory：Inference

Preparations

The following is required as a preliminary preparation.

  • Clone this repository to your local PC.
  • VoTT installation.
  • Google account(Used in Google Drive, Google Colaboratory)

1. VoTT：Annotation

Annotate using VoTT and output in TFRecord format.

VoTT project settings

Select "New Project"

2020-09-19 (3)

Make project settings

Display name：Tensorflow2-ObjectDetectionAPI-Colab-Hands-On
Security token：Generate New Security Token
Source connection：「Add Connection」を押下
2020-09-19 (4)

Set the connection of the source connection

Display name：Tensorflow2-ObjectDetectionAPI-Colab-Hands-On-TrainData 2020-09-19 (6) Provider: Local file system 2020-09-19 (7) Folder path：Specify the "01_train_data" directory of the cloned repository 2020-09-19 (8)

Set the connection of the target connection

Target connection：Add Connection 2020-09-19 (9) Display name：Tensorflow2-ObjectDetectionAPI-Colab-Hands-On-TFRecord
Provider: Local file system
Folder path：Specify the "02_tfrecord" directory of the cloned repository
2020-09-19 (10)

Add tags and save settings

Tags：Add "Fish"
Press "Save Project" 94047577-3d9dc080-fe0d-11ea-9f4f-b5fe7727fc12

Annotate using VoTT

Select a fish by left dragging the mouse

2020-09-19 (13)

Select "Fish" from TAGS

You can lock the tag you want to use by selecting the padlock mark. 2020-09-19 (14)

TFRecord export

Export settings

Provider：Tensorflow record
Asset status: Tagged assets only
Click "Save Export Settings" 2020-09-19 (16) Click the export icon from the annotation screen to export TFRecord. 2020-09-19 (14)

Precautions (details are being confirmed)

When annotating the target at the edge of the image, leave a small gap from the edge as shown below. 2020-09-19 (17)
I am checking the details of the problem, but if I annotate without opening a gap,
I do not know whether it is a VoTT problem or a Tensorflow problem, but the following error occurs when training the model.

W0921 13:29:32.965700 140050120722176 optimizer_v2.py:1275] Gradients do not exist for variables ['top_bn/gamma:0', 'top_bn/beta:0'] when minimizing the loss.
Traceback (most recent call last):
  File "object_detection/model_main_tf2.py", line 113, in 
    tf.compat.v1.app.run()
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 300, in run
    _run_main(main, args)
  File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "object_detection/model_main_tf2.py", line 110, in main
    record_summaries=FLAGS.record_summaries)
  File "/usr/local/lib/python3.6/dist-packages/object_detection/model_lib_v2.py", line 639, in train_loop
    loss = _dist_train_step(train_input_iter)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 780, in __call__
    result = self._call(*args, **kwds)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 807, in _call
    return self._stateless_fn(*args, **kwds)  # pylint: disable=not-callable
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 2829, in __call__
    return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 1848, in _filtered_call
    cancellation_manager=cancellation_manager)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 1924, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 550, in call
    ctx=ctx)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument:  assertion failed: [[0.15956609][0.103383526][0.109880842]...] [[0.23180081][0.133959055][0.132812485]...]
	 [[{{node Assert_1/AssertGuard/else/_35/Assert_1/AssertGuard/Assert}}]]
	 [[MultiDeviceIteratorGetNextFromShard]]
	 [[RemoteCall]]
	 [[IteratorGetNext]]
	 [[Loss/localization_loss_1/write_summary/summary_cond/pivot_t/_4/_111]]
  (1) Invalid argument:  assertion failed: [[0.15956609][0.103383526][0.109880842]...] [[0.23180081][0.133959055][0.132812485]...]
	 [[{{node Assert_1/AssertGuard/else/_35/Assert_1/AssertGuard/Assert}}]]
	 [[MultiDeviceIteratorGetNextFromShard]]
	 [[RemoteCall]]
	 [[IteratorGetNext]]

2. Colaboratory：Object Detection API Setting

Open In Colab
Subsequent work will be performed on Google Colaboratory. ※Except for pipeline config modification
Open your notebook from the [Open In Colab] link and run it in the following order.

  • Google Drive mount
  • Set Tensorflow Object Detection API
  • Clone Tensorflow2-ObjectDetectionAPI-Colab-Hands-On repository

3.Upload TFRecord

Store the TFRecord exported from VoTT and tf_label_map.pbtxt in "Tensorflow2-Object Detection API-Colab-Hands-On / 02_tfrecord".
After storing, execute the following.

  • Split Training data/validation data

4. Pipeline-config correction

Modify the pipeline config of "03_pretrained_model/coefficientdet_d0_coco17_tpu-32/pipeline.config" as follows,
Please upload to "Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/03_pretrained_model" on Colaboratory.

Pipeline-config correction part
  • 3行目(Line 3)：クラス数(num_classes)
    変更前(Before) : 90
    変更後(After) : 1
  • 134行目(Line 134)：バッチサイズ(batch_size)
    変更前(Before) : 128
    変更後(After) : 16
  • 161行目(Line 161)：ファインチューニング用のチェックポイント格納先(fine_tune_checkpoint)
    変更前(Before) : "PATH_TO_BE_CONFIGURED"
    変更後(After) : "/content/models/research/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/03_pretrained_model/efficientdet_d0_coco17_tpu-32/checkpoint/ckpt-0"
  • 167行目(Line 167)：ファインチューニング方法(fine_tune_checkpoint_type)
    変更前(Before) : "classification"
    変更後(After) : "detection"
  • 168行目(Line 168)：Googleカスタム 16ビットbrain浮動小数点の使用有無(use_bfloat16)
    変更前(Before) : true
    変更後(After) : false
  • 172行目(Line 172)：ラベルマップファイルの格納先(label_map_path)
    変更前(Before) : "PATH_TO_BE_CONFIGURED/label_map.txt"
    変更後(After) : "/content/models/research/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/02_tfrecord/tf_label_map.pbtxt"
  • 174行目(Line 174)：学習データの格納先(input_path)
    変更前(Before) : "PATH_TO_BE_CONFIGURED/train2017-?????-of-00256.tfrecord"
    変更後(After) : "/content/models/research/train_data/??????.tfrecord"
  • 185行目(Line 185)：ラベルマップファイルの格納先(label_map_path)
    変更前(Before) : "PATH_TO_BE_CONFIGURED/label_map.txt"
    変更後(After) : "/content/models/research/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/02_tfrecord/tf_label_map.pbtxt"
  • 189行目(Line 189)：バリデーションデータの格納先(input_path)
    変更前(Before) : "PATH_TO_BE_CONFIGURED/val2017-?????-of-00032.tfrecord"
    変更後(After) : "/content/models/research/val_data/??????.tfrecord"
Pipeline-config correction parts ※Those who can afford

Data Augmentation settings are also listed in the pipeline config.
In the initial pipeline config, the following horizontal inversion and random scale crop only data augmentation are set.

  data_augmentation_options {
    random_horizontal_flip {
    }
  }
  data_augmentation_options {
    random_scale_crop_and_pad_to_square {
      output_size: 512
      scale_min: 0.10000000149011612
      scale_max: 2.0
    }
  }

Available data augmentation techniques are preprocessor.proto、preprocessor.pyso
Try adding as needed.

5. Colaboratory：Model training

Please execute in the following order.

  • Create directory in Google Drive
  • TensorBoard
  • Training
  • Export to saved-model format

6. Colaboratory：Inference

Please execute in the following order.

  • Load model
  • Inference
  • Inference result confirmation

ToDo

Author

Kazuhito Takahashi(https://twitter.com/KzhtTkhs)

License

Tensorflow2-ObjectDetectionAPI-Colab-Hands-On is under MIT license.