Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support TFLite and TFJS conversion for CenterNet multi-class keypoints #11101

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

biantsh
Copy link

@biantsh biantsh commented Oct 20, 2023

Description

Rewrote the _postprocess_keypoints_multi_class() method in center_net_meta_arch.py to not use tf.tensor_scatter_nd_add ops, which seem to:

  • Break conversion to TFJS (Error message explicitly says TensorScatterAdd is not supported).
  • Not break conversion to TFLite, but inference will fail unless at least one object from all classes is predicted in any given output batch (see this issue raised in tensorflow/tensorflow).

This is a non-breaking change: The model's outputs in the exported SavedModel format remain 100% unchanged after this rewrite. Additionally, no negative impact on latency was observed - my tests yielded a latency improvement of ~3%:

Before rewrite After rewrite Improvement
Latency (batch size 1) 8.63 ms 8.37 ms 3.10%
Latency (batch size 32) 192.29 ms 185.54 ms 3.63%

For this reason, I replaced the code in the existing _postprocess_keypoints_multi_class() method. However, please let me know if I should leave that method as-is and add a new one instead.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Tests

Besides the unit test added in this PR, I ran tests to ensure that the TFLite and TFJS models run successfully without crashing, as well as to test outputs and latency of SavedModels. I have uploaded the models and Python scripts in the drive link below:

https://drive.google.com/drive/folders/1wCUvu_LQN6maRQFAKkAU8l1JnsOYm_-F?usp=sharing

Test Configuration: The scripts were ran in Python 3.10.13 and Tensorflow 2.8.1/2.14.0.

Checklist

@biantsh biantsh requested a review from jch1 as a code owner October 20, 2023 10:25
@biantsh
Copy link
Author

biantsh commented Oct 31, 2023

Hi @jch1, any updates on this? Is this repo still accepting contributions?

@laxmareddyp laxmareddyp added the models:research models that come under research directory label Jan 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
models:research models that come under research directory
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants