Skip to content

Latest commit

 

History

History
82 lines (56 loc) · 3.07 KB

README.md

File metadata and controls

82 lines (56 loc) · 3.07 KB
Raw

CBP

Official Tensorflow Implementation of the AAAI-2020 paper Temporally Grounding Language Queries in Videos by Contextual Boundary-aware Prediction by Jingwen Wang et al.

alt text

Citation

@inproceedings{wang2020temporally,
  title={Temporally Grounding Language Queries in Videos by Contextual Boundary-aware Prediction},
  author={Wang, Jingwen and Ma, Lin and Jiang, Wenhao},
  booktitle={AAAI},
  year={2020}
}

Requirements

pip install -r requirements.txt

Data Preparation

  1. Download Glove word embedding data.
cd download/
sh download_glove.sh
  1. Download dataset features.

TACoS: BaiduDrive, GoogleDrive

Charades-STA: BaiduDrive, GoogleDrive

ActivityNet-Captions: BaiduDrive, GoogleDrive

Put the feature hdf5 file in the corresponding directory ./datasets/{DATASET}/features/

We decode TACoS/Charades videos using fps=16 and extract C3D (fc6) features for each non-overlap 16-frame snippet. Therefore, each feature corresponds to 1-second snippet. For ActivityNet, each feature corresponds to 2-second snippet. To extract C3D fc6 features, I mainly refer to this code.

  1. Download trained models.

Download and put the checkpoints in corresponding ./checkpoints/{DATASET}/ .

BaiduDrive, GoogleDrive

  1. Data Preprocessing (Optional)
cd datasets/tacos/
sh prepare_data.sh

Then copy the generated data in ./data/save/ .

Use correspondig scripts for preparing data for other datasets.

You may skip this procedure as the prepared data is already saved in ./datasets/{DATASET}/data/save/ .

Testing and Evaluation

sh scripts/test_tacos.sh
sh scripts/eval_tacos.sh

Use corresponding scripts for testing or evaluating for other datasets.

The predicted results are also provided in ./results/{DATASET}/ .

CBP R@1,IoU=0.7 R@1,IoU=0.5 R@5,IoU=0.7 R@5,IoU=0.5 mIoU
TACoS 18.54 23.19 24.88 35.83 20.46
Charades 17.98 36.21 50.27 70.51 35.70
ActivityNet 18.74 36.83 49.84 67.78 37.98

Training

sh scripts/train_tacos.sh

Use corresponding scripts for training for other datasets.

Update

  1. The checkpoints for Charades dataset have been re-uploaded.