Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix Llama-template's system prompt bug #5156

Merged
merged 2 commits into from
Aug 19, 2024
Merged

fix Llama-template's system prompt bug #5156

merged 2 commits into from
Aug 19, 2024

Conversation

YeQiuO
Copy link
Contributor

@YeQiuO YeQiuO commented Aug 12, 2024

What does this PR do?

Llama 模型对应的 jinjia 模板有误,会导致系统提示被覆盖。

目前 Llama 的 jinjia 模板:

{% set system_message = '[template.default_system]' %}

{% if messages[0]['role'] == 'system' %}
    {% set system_message = messages[0]['content'] %}
{% endif %}

{% for message in messages %}
    {% set content = message['content'] %}
    
    {% if loop.index0 == 0 and system_message is defined %}
        {% set content = '<<SYS>>\n' + system_message + '\n<</SYS>>\n\n' + message['content'] %}
    {% endif %}
    
    {% if message['role'] == 'user' %}
        {{ '[INST] ' + content + ' [/INST]' }}
    {% elif message['role'] == 'assistant' %}
        {{ content + '' }}
    {% endif %}
{% endfor %}

该方法只能满足默认的system_message,用户自定义的 system_message 会被后一轮 user 覆盖。

为解决这一问题,修改了判断条件,自定义系统提示词时,跳过第一轮。

Fixes #5123

Before submitting

Copy link
Owner

@hiyouga hiyouga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hiyouga hiyouga merged commit 15be296 into hiyouga:main Aug 19, 2024
1 check passed
@hiyouga hiyouga added the solved This problem has been already solved label Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
solved This problem has been already solved
Projects
None yet
Development

Successfully merging this pull request may close these issues.

template使用问题
2 participants