-
Notifications
You must be signed in to change notification settings - Fork 1
/
prompt_v2.py
161 lines (92 loc) · 6.82 KB
/
prompt_v2.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
def fact_check_prompt(question, annotation, language):
cn_user_prompt = f"""
你将作为一个事实判断器，我会给你提供一个问题和一个针对该问题的部分回答，你的任务是判断回答中的内容是否存在可以判断的事实。
## 判断标准：
- **可以判断的事实：** 具体的、客观的信息点，这些信息可以通过数据、研究结果或其他可靠来源进行验证。例如，统计数据、历史事件、科学定律、具体案例等。
- **非事实描述：** 个人意见、主观判断或无法验证的声明。
## 任务流程：
### 1. **仔细阅读问题，问题如下：**
{question}
### 2. **仔细阅读回答，部分回答如下：**
{annotation}
### 3. **进行分析：** 根据上述判断标准，判断回答中是否包含可以判断的事实。
- 如果回答中不存在可以判断的事实，则输出“<无事实>”。
- 如果回答中存在可以判断的事实，则输出“<有事实>”。
"""
en_user_prompt = f"""
You will act as a fact checker, and I will provide you with a question and a corresponding partial answer. Your task is to determine whether the content of the answer contains verifiable facts.
## Judgment Criteria:
- **Verifiable Facts:** Specific, objective points of information that can be verified through data, research results, or other reliable sources. Examples include statistical data, historical events, scientific laws, and specific case studies.
- **Non-factual Descriptions:** Personal opinions, subjective judgments, or unverifiable statements.
## Task Process:
### 1. **Carefully read the question, which is as follows:**
{question}
### 2. **Carefully read the partial answer, which is as follows:**
{annotation}
### 3. **Conduct the Analysis:** Based on the above judgment criteria, determine if the answer contains verifiable facts.
- If there are no verifiable facts in the answer, output “<No Facts>”.
- If there are verifiable facts in the answer, output “<Facts Present>”.
"""
return cn_user_prompt if language == "zh" else en_user_prompt
def reference_check_prompt(question, reference, annotation, language):
cn_user_prompt = f"""
你将作为一个信息提取器，我将给你提供一个问题、一份相关的参考文档，以及一个针对该问题的部分回答，你的任务是从参考文档中提炼出与问题和回答相关的信息。
## 操作步骤：
### 1. **仔细阅读问题，问题如下：**
{question}
### 2. **仔细阅读回答，部分回答如下：**
{annotation}
### 3. **分析参考文档：** 找出与问题和回答最相关的信息，这些信息可能与回答内容完全相同、部分相同，或存在冲突。
**参考文档如下：**
{reference}
### 4. **列出相关信息：** 按顺序列出所有发现的相关信息，如果有多条信息的话以 <SEP> 作为分隔。
### 5. **无相关信息时输出：** 如果没有找到相关信息，请输出<无参考信息>。
"""
en_user_prompt = f"""
You will act as an information extractor. I will provide you with a question, a related reference document, and a partial answer to that question. Your task is to extract information from the reference document that is relevant to the question and answer.
## Operational Steps:
### 1. **Carefully read the question, which is as follows:**
{question}
### 2. **Carefully read the partial answer, which is as follows:**
{annotation}
### 3. **Analyze the Reference Document:** Identify information most relevant to the question and answer. This information may be completely the same, partially similar, or conflicting with the content of the answer.
**The reference document is as follows:**
{reference}
### 4. **List the Relevant Information:** List all the relevant information found in order, separated by <SEP> if there are multiple pieces of information.
### 5. **Output When No Information Is Found:** If no relevant information is found, output <No Reference Information>.
"""
return cn_user_prompt if language == "zh" else en_user_prompt
def hallucination_check_prompt(question, reference, annotation, language):
cn_user_prompt = f"""
你将作为一个‘幻觉’标注器，我将会给你提供一个一个问题，一个针对该问题的部分回答和相关的参考要点。你需要判断提供的回答中是否含有幻觉性内容，并标注幻觉类型。
‘幻觉’指的是与参考要点相矛盾或在参考要点中没有依据的内容。
## 判断准则：
1. **无幻觉：** 如果回答与参考要点完全一致，且没有引入与参考要点相矛盾的信息，请输出：<无幻觉>。
2. **矛盾：** 如果回答内容与参考要点存在明显矛盾，请输出：<矛盾>。
3. **无法验证：** 如果回答包含的信息在参考要点中没有提及，且无法从参考要点中得到支持或验证，请输出：<无法验证>。
## 任务流程：
### 1. **仔细阅读问题，问题如下：**
{question}
### 2. **仔细阅读回答，部分回答如下：**
{annotation}
### 3. **仔细阅读参考要点，参考要点如下：**
{reference}
### 4. **进行分析：** 根据上述判断标准，判断回答中是否包含幻觉，并输出幻觉类型。
"""
en_user_prompt = f"""
You will act as a 'Hallucination' annotator. I will provide you with a question, a partial answer to that question, and related reference points. You need to determine whether the provided answer contains any hallucinatory content and annotate the type of hallucination.
'Hallucination' refers to content that contradicts the reference points or is unsupported by them.
## Judgment Criteria:
1. **No Hallucination:** If the answer is completely consistent with the reference points and does not introduce any contradictory information, output: <No Hallucination>.
2. **Contradiction:** If the answer clearly contradicts the reference points, output: <Contradictory>.
3. **Unverifiable:** If the answer contains information not mentioned in the reference points and cannot be supported or verified by them, output: <Unverifiable>.
## Task Process:
### 1. **Carefully read the question, which is as follows:**
{question}
### 2. **Carefully read the partial answer, which is as follows:**
{annotation}
### 3. **Carefully read the reference points, which are as follows:**
{reference}
### 4. **Conduct the analysis:** Based on the above judgment criteria, determine if the answer contains hallucinations and output the type of hallucination.
"""
return cn_user_prompt if language == "zh" else en_user_prompt