From af7e289adc01efd12962d06c0b2fae725b69cf1b Mon Sep 17 00:00:00 2001 From: Shivraj Date: Mon, 11 Mar 2019 14:19:05 -0700 Subject: [PATCH 01/10] Create README.md --- mrcnn/README.md | 83 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 mrcnn/README.md diff --git a/mrcnn/README.md b/mrcnn/README.md new file mode 100644 index 0000000..8dc0c98 --- /dev/null +++ b/mrcnn/README.md @@ -0,0 +1,83 @@ +# Data Synthesis + +## Table of Contents + +1. [Introduction](#introduction) +1. [Examples](#examples) +1. [Usage](#usage) + + +## Introduction + + +## Setup + +Run data syntheszer module to generate the training datasets in the following folder structure: + +~~~~~~~ + project + |-- mrcnn + |-- scipts + |-- config.py + |-- model.py + |-- train.py + |-- pre_process.py + |-- requirements.txt + |-- utils.py + |-- visualize.py + |-- Inference_notebook.ipynb + |-- utils.py + + |--- mask_data + |-- id_map.json + |-- logs/ + |-- mask_rcnn_coco.h5 + |-- test_image + |-- train_image +~~~~~~~ + +## Training + + +### Pre-processing + + +### Model and Training + +``` + -- Modified Matterport's implementation of Mask-RCNN deep neural network for object instance segmentation. + -- Increased maximum number of predicted objects since an image can contain 200 or more bottles. + -- Increased POST_NMS_ROIS_TRAINING to get more region proposals during training. + -- Resized images and masks to 512x512 + -- Relied heavily on deep image augmentation due to small training set: + Random horizontal or vertical flips + Random 90 or -90 degrees rotation + Random rotations in the range of (-20, 20) degrees + Random scaling of image and mask scaling in the range (0.5, 2.0) + -- Used Resnet101 architecture as a backbone encoder. + -- Trained the model with Adam optimizer for 75 epochs: + -- 5 epochs of heads with learning rate 1e-4 + -- 30 epochs with learning rate 1e-5 + -- 30 epochs with learning rate 1e-6 + -- changed mAP computation to 0.5 - 0.8 + -- weighted mAP +``` + +Run python pre_process.py to pre-process data + +Run python train.py to train the model. Model weights are saved at ../data/logs/kaggle_bowl/mask_rcnn.h5. + +Run python inference_notebook.ipynb.py to evaluate model performance on test set + +### Model Execution Time + +The following execution times are measured on Nvidia P100 GPUs provided by AWS Deep learning AMI + +``` +Each training epoch takes about 25 minutes. +It takes about 18 hours to train the model from scratch. +``` + +## Example model predictions + +[put graphs from notebook] From ef3050b31f203c31a74f30ba8a695bbc9069b33d Mon Sep 17 00:00:00 2001 From: Shivraj Date: Wed, 13 Mar 2019 14:28:20 -0700 Subject: [PATCH 02/10] Added more inputs to model training --- mrcnn/README.md | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-) diff --git a/mrcnn/README.md b/mrcnn/README.md index 8dc0c98..93e84ef 100644 --- a/mrcnn/README.md +++ b/mrcnn/README.md @@ -40,36 +40,51 @@ Run data syntheszer module to generate the training datasets in the following f ### Pre-processing +Data pre-processing using pre_process.py to generate .h5 file for masks. ### Model and Training ``` -- Modified Matterport's implementation of Mask-RCNN deep neural network for object instance segmentation. - -- Increased maximum number of predicted objects since an image can contain 200 or more bottles. + Model - Added two new methods to train just specific masks. + mrcnn_mask: Just mask layers + mask_heads: Mask layers or rpn/fpn + Multiclass- Pre_processed images and mask files accordingly to prepare for multi classification. + -- Increased maximum number of predicted objects since an image can contain 200 or more bottles/bags/boxes. -- Increased POST_NMS_ROIS_TRAINING to get more region proposals during training. - -- Resized images and masks to 512x512 + -- Resized images and masks to 512x512. + -- Used Default anchor size as we do not expect small objects: RPN_ANCHOR_SCALES = (16, 32, 64, 128, 256) -- Relied heavily on deep image augmentation due to small training set: Random horizontal or vertical flips Random 90 or -90 degrees rotation Random rotations in the range of (-20, 20) degrees Random scaling of image and mask scaling in the range (0.5, 2.0) -- Used Resnet101 architecture as a backbone encoder. - -- Trained the model with Adam optimizer for 75 epochs: - -- 5 epochs of heads with learning rate 1e-4 + -- Trained the model with Adam optimizer for 65 epochs: + -- 5 epochs of heads with learning rate 1e-4 (To speed up the training process) -- 30 epochs with learning rate 1e-5 -- 30 epochs with learning rate 1e-6 - -- changed mAP computation to 0.5 - 0.8 + -- changed mAP computation to be (0.5 - 0.8) -- weighted mAP + -- weighted loss + LOSS_WEIGHTS = { + "rpn_class_loss": 20., + "rpn_bbox_loss": 1., + "mrcnn_class_loss": 10., + "mrcnn_bbox_loss": 1., + "mrcnn_mask_loss": 10. + } + ``` - + +### Model Execution and Run-Time Run python pre_process.py to pre-process data Run python train.py to train the model. Model weights are saved at ../data/logs/kaggle_bowl/mask_rcnn.h5. Run python inference_notebook.ipynb.py to evaluate model performance on test set -### Model Execution Time The following execution times are measured on Nvidia P100 GPUs provided by AWS Deep learning AMI From 884fa360ec3b4fe132d7d3ca3bece87962153a96 Mon Sep 17 00:00:00 2001 From: Shivraj Date: Wed, 13 Mar 2019 14:29:15 -0700 Subject: [PATCH 03/10] Update header --- mrcnn/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mrcnn/README.md b/mrcnn/README.md index 93e84ef..650fde6 100644 --- a/mrcnn/README.md +++ b/mrcnn/README.md @@ -1,4 +1,4 @@ -# Data Synthesis +# Model Training ## Table of Contents From 3006406426ed7fef7e341f2a4b234e55ef7dc2dc Mon Sep 17 00:00:00 2001 From: Shivraj Date: Thu, 11 Jul 2019 12:28:39 -0700 Subject: [PATCH 04/10] Update mrcnn/README.md Co-Authored-By: Havan Agrawal --- mrcnn/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mrcnn/README.md b/mrcnn/README.md index 650fde6..e5ddd61 100644 --- a/mrcnn/README.md +++ b/mrcnn/README.md @@ -81,7 +81,7 @@ Data pre-processing using pre_process.py to generate .h5 file for masks. ### Model Execution and Run-Time Run python pre_process.py to pre-process data -Run python train.py to train the model. Model weights are saved at ../data/logs/kaggle_bowl/mask_rcnn.h5. +Run `python train.py` to train the model. Model weights are saved at ../data/logs/kaggle_bowl/mask_rcnn.h5. Run python inference_notebook.ipynb.py to evaluate model performance on test set From 77a36f27b9117f3ddcaa2cb9a451182dfecfde9c Mon Sep 17 00:00:00 2001 From: Shivraj Date: Thu, 11 Jul 2019 12:29:55 -0700 Subject: [PATCH 05/10] Fixed Typo --- mrcnn/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mrcnn/README.md b/mrcnn/README.md index e5ddd61..a9d9ab8 100644 --- a/mrcnn/README.md +++ b/mrcnn/README.md @@ -12,7 +12,7 @@ ## Setup -Run data syntheszer module to generate the training datasets in the following folder structure: +Run data synthesizer module to generate the training datasets in the following folder structure: ~~~~~~~ project From 12d29f79f050ed28d0dccdbb51c162ee39459260 Mon Sep 17 00:00:00 2001 From: Shivraj Date: Thu, 11 Jul 2019 12:30:18 -0700 Subject: [PATCH 06/10] Update mrcnn/README.md Co-Authored-By: Havan Agrawal --- mrcnn/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/mrcnn/README.md b/mrcnn/README.md index a9d9ab8..b38ad97 100644 --- a/mrcnn/README.md +++ b/mrcnn/README.md @@ -45,7 +45,6 @@ Data pre-processing using pre_process.py to generate .h5 file for masks. ### Model and Training -``` -- Modified Matterport's implementation of Mask-RCNN deep neural network for object instance segmentation. Model - Added two new methods to train just specific masks. mrcnn_mask: Just mask layers From 760b1585faadfeb6c854694128fd24a48a231f9d Mon Sep 17 00:00:00 2001 From: Shivraj Date: Thu, 11 Jul 2019 12:30:25 -0700 Subject: [PATCH 07/10] Update mrcnn/README.md Co-Authored-By: Havan Agrawal --- mrcnn/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mrcnn/README.md b/mrcnn/README.md index b38ad97..863e93b 100644 --- a/mrcnn/README.md +++ b/mrcnn/README.md @@ -78,7 +78,7 @@ Data pre-processing using pre_process.py to generate .h5 file for masks. ``` ### Model Execution and Run-Time -Run python pre_process.py to pre-process data +Run `python pre_process.py` to pre-process data Run `python train.py` to train the model. Model weights are saved at ../data/logs/kaggle_bowl/mask_rcnn.h5. From 0bfc2e6a62034d02f746079370503b56a64cebf9 Mon Sep 17 00:00:00 2001 From: Shivraj Date: Thu, 11 Jul 2019 12:30:40 -0700 Subject: [PATCH 08/10] Update mrcnn/README.md Co-Authored-By: Havan Agrawal --- mrcnn/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/mrcnn/README.md b/mrcnn/README.md index 863e93b..274113e 100644 --- a/mrcnn/README.md +++ b/mrcnn/README.md @@ -90,7 +90,6 @@ The following execution times are measured on Nvidia P100 GPUs provided by AWS D ``` Each training epoch takes about 25 minutes. It takes about 18 hours to train the model from scratch. -``` ## Example model predictions From ebb1444e99e82b8038d36101995b388b5b83ca50 Mon Sep 17 00:00:00 2001 From: Shivraj Date: Thu, 11 Jul 2019 12:34:41 -0700 Subject: [PATCH 09/10] Fixed bullet points --- mrcnn/README.md | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/mrcnn/README.md b/mrcnn/README.md index 274113e..48fc246 100644 --- a/mrcnn/README.md +++ b/mrcnn/README.md @@ -45,29 +45,29 @@ Data pre-processing using pre_process.py to generate .h5 file for masks. ### Model and Training - -- Modified Matterport's implementation of Mask-RCNN deep neural network for object instance segmentation. + * Modified Matterport's implementation of Mask-RCNN deep neural network for object instance segmentation. Model - Added two new methods to train just specific masks. mrcnn_mask: Just mask layers mask_heads: Mask layers or rpn/fpn Multiclass- Pre_processed images and mask files accordingly to prepare for multi classification. - -- Increased maximum number of predicted objects since an image can contain 200 or more bottles/bags/boxes. - -- Increased POST_NMS_ROIS_TRAINING to get more region proposals during training. - -- Resized images and masks to 512x512. - -- Used Default anchor size as we do not expect small objects: RPN_ANCHOR_SCALES = (16, 32, 64, 128, 256) - -- Relied heavily on deep image augmentation due to small training set: - Random horizontal or vertical flips - Random 90 or -90 degrees rotation - Random rotations in the range of (-20, 20) degrees - Random scaling of image and mask scaling in the range (0.5, 2.0) - -- Used Resnet101 architecture as a backbone encoder. - -- Trained the model with Adam optimizer for 65 epochs: - -- 5 epochs of heads with learning rate 1e-4 (To speed up the training process) - -- 30 epochs with learning rate 1e-5 - -- 30 epochs with learning rate 1e-6 - -- changed mAP computation to be (0.5 - 0.8) - -- weighted mAP - -- weighted loss - LOSS_WEIGHTS = { + * Increased maximum number of predicted objects since an image can contain 200 or more bottles/bags/boxes. + * Increased POST_NMS_ROIS_TRAINING to get more region proposals during training. + * Resized images and masks to 512x512. + * Used Default anchor size as we do not expect small objects: RPN_ANCHOR_SCALES = (16, 32, 64, 128, 256) + * Relied heavily on deep image augmentation due to small training set: + * Random horizontal or vertical flips + * Random 90 or -90 degrees rotation + * Random rotations in the range of (-20, 20) degrees + * Random scaling of image and mask scaling in the range (0.5, 2.0) + * Used Resnet101 architecture as a backbone encoder. + * Trained the model with Adam optimizer for 65 epochs: + * 5 epochs of heads with learning rate 1e-4 (To speed up the training process) + * 30 epochs with learning rate 1e-5 + * 30 epochs with learning rate 1e-6 + * changed mAP computation to be (0.5 - 0.8) + * weighted mAP + * weighted loss + * LOSS_WEIGHTS = { "rpn_class_loss": 20., "rpn_bbox_loss": 1., "mrcnn_class_loss": 10., From 9212d655e4b55906fa94043400bd2af7e58b719c Mon Sep 17 00:00:00 2001 From: Shivraj Date: Thu, 11 Jul 2019 13:28:26 -0700 Subject: [PATCH 10/10] Update README.md --- mrcnn/README.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/mrcnn/README.md b/mrcnn/README.md index 48fc246..269010a 100644 --- a/mrcnn/README.md +++ b/mrcnn/README.md @@ -3,8 +3,7 @@ ## Table of Contents 1. [Introduction](#introduction) -1. [Examples](#examples) -1. [Usage](#usage) +1. [Training](#training) ## Introduction @@ -75,9 +74,9 @@ Data pre-processing using pre_process.py to generate .h5 file for masks. "mrcnn_mask_loss": 10. } -``` ### Model Execution and Run-Time + Run `python pre_process.py` to pre-process data Run `python train.py` to train the model. Model weights are saved at ../data/logs/kaggle_bowl/mask_rcnn.h5.