有没有int4量化版，int4量化版推理需要多少什么显卡配置

[bbeyondllove]

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[YMMF007]

https://modelscope.cn/models/OPEA/DeepSeek-V3-int4-sym-gptq-inc/summary
7*80G

[bbeyondllove]

https://modelscope.cn/models/OPEA/DeepSeek-V3-int4-sym-gptq-inc/summary 7*80G

谢谢。那需要7张A100显卡了，成本比较高

[intelyoungway]

GPTQ可能会和calibrate数据集相关，感觉还是AWQ的好一些？
https://modelscope.cn/models/OPEA/DeepSeek-V3-int4-sym-awq-inc-cpu

[yishibakaien]

https://modelscope.cn/models/OPEA/DeepSeek-V3-int4-sym-gptq-inc/summary 7*80G

谢谢。那需要7张A100显卡了，成本比较高

英伟达 新出的 project digits 128g 通用 vram，4 台可以部署，单价 $3000

[bbeyondllove]

GPTQ可能会和calibrate数据集相关，感觉还是AWQ的好一些？ https://modelscope.cn/models/OPEA/DeepSeek-V3-int4-sym-awq-inc-cpu

好的，谢谢

[bbeyondllove]

https://modelscope.cn/models/OPEA/DeepSeek-V3-int4-sym-gptq-inc/summary 7*80G

谢谢。那需要7张A100显卡了，成本比较高

英伟达 新出的 project digits 128g 通用 vram，4 台可以部署，单价 $3000

如果用v2的量化版要求是不是低一点

[chenatu]

@YMMF007 请问这个配置下推理速度怎么样，单token耗时大概多少

[YMMF007]

@YMMF007 请问这个配置下推理速度怎么样，单token耗时大概多少

我这边显卡不够，就没跑了