Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Annotation maker with Blender and CLIP Interrogator or BLIP #2

Open
lalalune opened this issue Dec 5, 2022 · 2 comments
Open

Annotation maker with Blender and CLIP Interrogator or BLIP #2

lalalune opened this issue Dec 5, 2022 · 2 comments

Comments

@lalalune
Copy link

lalalune commented Dec 5, 2022

The datasets we have are decent model-wise, very low quality annotation wise.

I have experimented with running them through BLIP with really good success. With a 3D model on a black background, it pretty much always gets it right.

I've made a headless Blender script to capture the other data that we need:
https://github.com/nv-tlabs/GET3D/pull/17/files#diff-c95401620d13c6e6aed785545efca7bda6dbdc91c68ed5bcec4cb9947f1f7bbf

This could be extracted to a new repo (this one?) and we can add automatic annotation, so it can just be a singular pipeline.

@lalalune
Copy link
Author

lalalune commented Dec 7, 2022

https://github.com/lalalune/ImprovedShapenetRenderer

I've split the ShapeNet Renderer PR into a standalone repo.

@rom1504 what is SOTA for CLIP interrogation? Have been using https://github.com/salesforce/BLIP with great success on Shapenet, since we are rendering on a black background it's fairly accurate and low noise

@rom1504
Copy link
Contributor

rom1504 commented Dec 9, 2022

I think try out the specific blip pipeline we used for laion coco. Links are in blogpost

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants