This is an example of folder structure in the Lightly format for data selection using data from the DeepSportRadar Instance Segmentation Challenge! Predictions are computed on the union of val and test sets using a detectron2 mask_rcnn_R_50_FPN_3x model finetuned on the train set. Metadata are created with the name of the folders in which images are contained. KS-FR-ROANNE folder is removed using our relevant filenames feature. In this example you will find an implementation using AWS S3. Lightly also supports working with Azure and with Google Cloud Storage. You need to adjust some code accordingly to the docs in order to use these other providers, but the folder structure remains the same.
This is how the folder structure looks like:
.
├── input_bucket/
│ ├── KS-FR-BLOIS/
│ │ ├── 24330/
│ │ │ ├── camcourt1_1513714448590_0.png
│ │ │ └── ...
│ │ └── ...
│ └── ...
└── lightly_bucket/
└── .lightly/
├── metadata/
│ ├── schema.json
│ ├── KS-FR-BLOIS/
│ │ ├── 24330/
│ │ │ ├── camcourt1_1513714448590_0.png
│ │ │ └── ...
│ │ └── ...
│ └── ...
├── predictions/
│ ├── tasks.json
│ └── object_detection_sportradar/
│ ├── schema.json
│ ├── KS-FR-BLOIS/
│ │ ├── 24330/
│ │ │ ├── camcourt1_1513714448590_0.png
│ │ │ └── ...
│ │ └── ...
│ └── ...
└── relevant_filenames.txt
In order to run the selection (on valtest set) you have first to upload data to your cloud buckets by using the following AWS CLI commands:
aws s3 cp input_bucket s3://yourInputBucket/sportradar_input/ --recursive
aws s3 cp lightly_bucket s3://yourLightlyBucket/sportradar_lightly/ --recursive
If you already installed Lightly, run the worker with your API token and your worker id:
docker run --shm-size="1024m" --gpus all --rm -it \
-e LIGHTLY_TOKEN={MY_LIGHTLY_TOKEN} \
lightly/worker:latest \
worker.worker_id={MY_WORKER_ID}
And finally run your Python scheduling script with:
python run_lightly.py