Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support large ONNX model files #828

Merged
merged 14 commits into from
Sep 26, 2022
Merged

feat: support large ONNX model files #828

merged 14 commits into from
Sep 26, 2022

Conversation

ZiniuYu
Copy link
Member

@ZiniuYu ZiniuYu commented Sep 22, 2022

NOTICE Please merge #825 first!!!

This PR supports the following ONNX models:

  • ViT-H-14::laion2B-s32B-b79K
  • ViT-g-14::laion2B-s12B-b42K
  • M-CLIP/XLM-Roberta-Large-Vit-B-32
  • M-CLIP/XLM-Roberta-Large-Vit-L-14
  • M-CLIP/XLM-Roberta-Large-Vit-B-16Plus
  • M-CLIP/LABSE-Vit-L-14

This is accomplished by rewriting the loading logic: if the model file is a zip file, we unzip it to a tmp dir and load the model from there.

This PR also rewrites loading M-CLIP logic: the visual model now loads from our custom open-clip models

@codecov
Copy link

codecov bot commented Sep 22, 2022

Codecov Report

Merging #828 (4b754e3) into main (09d1548) will increase coverage by 1.80%.
The diff coverage is 52.94%.

@@            Coverage Diff             @@
##             main     #828      +/-   ##
==========================================
+ Coverage   82.57%   84.38%   +1.80%     
==========================================
  Files          21       21              
  Lines        1567     1575       +8     
==========================================
+ Hits         1294     1329      +35     
+ Misses        273      246      -27     
Flag Coverage Δ
cas 84.38% <52.94%> (+1.80%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
server/clip_server/model/clip_onnx.py 78.18% <41.66%> (-10.71%) ⬇️
server/clip_server/model/mclip_model.py 83.33% <80.00%> (-0.88%) ⬇️
server/clip_server/model/clip_model.py 91.66% <0.00%> (+4.16%) ⬆️
server/clip_server/model/clip_trt.py 85.71% <0.00%> (+16.32%) ⬆️
server/clip_server/model/trt_utils.py 83.51% <0.00%> (+27.47%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@ZiniuYu ZiniuYu linked an issue Sep 23, 2022 that may be closed by this pull request
@ZiniuYu ZiniuYu marked this pull request as ready for review September 23, 2022 04:58
@ZiniuYu ZiniuYu requested a review from a team September 23, 2022 04:59
@numb3r3 numb3r3 requested a review from jemmyshin September 23, 2022 07:24
@ZiniuYu ZiniuYu changed the title feat: support not yet supported onnx models feat: support large ONNX model files Sep 23, 2022
server/clip_server/model/clip_onnx.py Outdated Show resolved Hide resolved
server/clip_server/model/clip_onnx.py Outdated Show resolved Hide resolved
@github-actions
Copy link

📝 Docs are deployed on https://ft-zip-model--jina-docs.netlify.app 🎉

@ZiniuYu ZiniuYu requested a review from a team September 26, 2022 02:45
@numb3r3 numb3r3 merged commit 2ba8a4f into main Sep 26, 2022
@numb3r3 numb3r3 deleted the zip-model branch September 26, 2022 02:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Onyx + TensorRT support for XLM-R Large Vit-B/16+?
3 participants