Mac系统下部署安装完成的问题

[zhug-e]

Reminder

• I have read the README and searched the existing issues.

Reproduction

1

Expected behavior

1

System Info

M1芯片 MacOS 14.5

Others

启动加载模型都成功，在问答的时候报错如下：
Exception in thread Thread-8:
Traceback (most recent call last):
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "/Library/Python/3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/Users/zhugang/Library/Python/3.9/lib/python/site-packages/transformers/generation/utils.py", line 1569, in generate
model_kwargs["attention_mask"] = self._prepare_attention_mask_for_generation(
File "/Users/zhugang/Library/Python/3.9/lib/python/site-packages/transformers/generation/utils.py", line 468, in _prepare_attention_mask_for_generation
raise ValueError(
ValueError: Can't infer missing attention mask on mps device. Please provide an attention_mask or use a different device.

[DarthPenguinz]

@zhug-e Hi, any fixes for this yet? am facing the same issue.

[gklab]

1. find File "/Users/zhugang/Library/Python/3.9/lib/python/site-packages/transformers/generation/utils.py", line 468
2. Comment out this detection judgment

It can be executed normally with the latest mps version of pytorch.

[zhug-e]

Thank you very much for your help, according to this method is effective! 发自我的iPhone

…

------------------ 原始邮件 ------------------ 发件人: ZHAOKAI WANG ***@***.***> 发送时间: 2024年5月26日 17:17 收件人: hiyouga/LLaMA-Factory ***@***.***> 抄送: zhug-e ***@***.***>, Mention ***@***.***> 主题: Re: [hiyouga/LLaMA-Factory] Mac系统下部署安装完成的问题 (Issue #3873) find File "/Users/zhugang/Library/Python/3.9/lib/python/site-packages/transformers/generation/utils.py", line 468 Comment out this detection judgment It can be executed normally with the latest mps version of pytorch. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[hiyouga]

fixed