防护栏是一组规则和检查机制，旨在确保大语言模型的输出准确、恰当并符合用户预期。如需了解更多关于开发防护栏的信息，可以参考这份防护栏开发指南。
在本笔记本中，我们将逐步开发一个输出防护栏，专门用于检查模型输出的幻觉问题。
本笔记本将重点关注：
- 构建一个强大的评估集
- 确定衡量幻觉的具体标准
- 通过少量样本提示提高我们防护栏的准确性
from concurrent.futures import ThreadPoolExecutor
from IPython.display import display, HTML
import json
import pandas as pd
from sklearn.metrics import precision_score, recall_score
from typing import List
from openai import OpenAI
client = OpenAI()# Function to set up display options for pandas
def setup_pandas_display():
# Increase display limits
pd.set_option('display.max_rows', 500)
pd.set_option('display.max_columns', 500)
# Function to make DataFrame scrollable in the notebook output
def make_scrollable(df):
style = (
'<style>'
'div.output_scroll {'
'resize: both;'
'overflow: auto;'
'}'
'</style>'
)
html = f"{style}{df.to_html()}"
display(HTML(html))
# Main function to display DataFrame
def display_dataframe(df):
setup_pandas_display() # Enable scrollable view
make_scrollable(df)
1. 构建评估集
假设我们是一个客户支持团队，正在构建一个自动化支持智能体。我们将向助手提供来自知识库的特定政策信息，例如如何处理退货、退款、反馈等工单，并期望模型在与客户互动时遵循这些政策。
我们要做的第一件事是使用GPT-4o来制定一套我们希望遵循的策略。
如果你想深入了解生成合成数据，可以查阅我们的合成数据生成指南此处
system_input_prompt = """
You are a helpful assistant that can generate policies for a support agent at a fictional company to follow. You will be provided with a topic (ie. returns, refunds, feedback) and you are to generate a sample policy for how to handle the it.
When constructing the policy, it should contain step-by-step instructions for how to handle the customer inquiry. It should include decision logic for what to do if a customer falls under a certain category, and provide requirements for taking specific actions.
"""
user_policy_example_1 = """"
RETURN POLICY
"""
assistant_policy_example_1 = """
RETURN POLICY
1. ** Ask the customer why they want the order replaced **
- Categorize their issue into one of the following buckets:
- damaged: They received the product in a damaged state
- satisfaction: The customer is not satisfied with the item and does not like the product.
- unnecessary: They no longer need the item
2a. **If return category is 'damaged'
- Ask customer for a picture of the damaged item
- If the item is indeed damaged, continue to step 3
- If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund
- Skip step 3 and go straight to step 4
2b. **If return category is either 'satisfaction' or 'unnecessary'**
- Ask the customer if they can provide feedback on the quality of the item
- If the order was made within 30 days, notify them that they are eligible for a full refund
- If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%
- If the order was made greater than 60 days ago, notify them that they are not eligible for a refund
3. **If the customer is eligible for a return or refund**
- Ask the customer to confirm that they would like a return or refund
- Once they confirm, process their request
4 **Provide additional support before closing out ticket**
- Ask the customer if there is anything else you can do to help them today.
"""
user_policy_input = """
{{POLICY}}
"""def generate_policy(policy: str) -> str:
input_message = user_policy_input.replace("{{POLICY}}", policy)
response = client.chat.completions.create(
messages= [
{"role": "system", "content": system_input_prompt},
{"role": "user", "content": user_policy_example_1},
{"role": "assistant", "content": assistant_policy_example_1},
{"role": "user", "content": input_message},
],
model="gpt-4o"
)
return response.choices[0].message.content
def generate_policies() -> List[str]:
# List of different types of policies to generate
policies = ['PRODUCT FEEDBACK POLICY', 'SHIPPING POLICY', 'WARRANTY POLICY', 'ACCOUNT DELETION', 'COMPLAINT RESOLUTION']
with ThreadPoolExecutor() as executor:
policy_instructions_list = list(executor.map(generate_policy, policies))
return policy_instructions_list
policy_instructions = generate_policies()接下来我们将根据这些策略生成遵循或未遵循指示的客户互动样本。
system_input_prompt = """"
You are a helpful assistant that can generate fictional interactions between a support assistant and a customer user. You will be given a set of policy instructions that the support agent is instructed to follow.
Based on the instructions, you must generate a relevant single-turn or multi-turn interaction between the assistant and the user. It should average between 1-3 turns total.
For a given set of instructions, generate an example conversation that where the assistant either does or does not follow the instructions properly. In the assistant's responses, have it give a combination of single sentence and multi-sentence responses.
The output must be in a json format with the following three parameters:
- accurate:
- This should be a boolean True or False value that matches whether or not the final assistant message accurately follows the policy instructions
- kb_article:
- This should be the entire policy instruction that is passed in from the user
- chat_history:
- This should contain the entire conversation history except for the final assistant message.
- This should be in a format of an array of jsons where each json contains two parameters: role, and content.
- Role should be set to either 'user' to represent the customer, or 'assistant' to represent the customer support assistant.
- Content should contain the message from the appropriate role.
- The final message in the chat history should always come from the user. The assistant response in the following parameter will be a response to this use message.
- assistant_response:
- This should contain the final response from the assistant. This is what we will evaluate to determine whether or not it is accurately following the policy.
"""
user_example_1 = """"
Here are the policy instructions:
RETURN POLICY
1. ** Ask the customer why they want the order replaced **
- Categorize their issue into one of the following buckets:
- damaged: They received the product in a damaged state
- satisfaction: The customer is not satisfied with the item and does not like the product.
- unnecessary: They no longer need the item
2a. **If return category is 'damaged'
- Ask customer for a picture of the damaged item
- If the item is indeed damaged, continue to step 3
- If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund
- Skip step 3 and go straight to step 4
2b. **If return category is either 'satisfaction' or 'unnecessary'**
- Ask the customer if they can provide feedback on the quality of the item
- If the order was made within 30 days, notify them that they are eligible for a full refund
- If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%
- If the order was made greater than 60 days ago, notify them that they are not eligible for a refund
3. **If the customer is eligible for a return or refund**
- Ask the customer to confirm that they would like a return or refund
- Once they confirm, process their request
4 **Provide additional support before closing out ticket**
- Ask the customer if there is anything else you can do to help them today.
"""
assistant_example_1 = """
{
"accurate": "true",
"kb_article": "1. ** Ask the customer why they want the order replaced ** - Categorize their issue into one of the following buckets: - damaged: They received the product in a damaged state - satisfaction: The customer is not satisfied with the item and does not like the product. - unnecessary: They no longer need the item 2a. **If return category is 'damaged' - Ask customer for a picture of the damaged item - If the item is indeed damaged, continue to step 3 - If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund - Skip step 3 and go straight to step 4 2b. **If return category is either 'satisfaction' or 'unnecessary'** - Ask the customer if they can provide feedback on the quality of the item - If the order was made within 30 days, notify them that they are eligible for a full refund - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50% - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund 3. **If the customer is eligible for a return or refund** - Ask the customer to confirm that they would like a return or refund - Once they confirm, process their request 4 **Provide additional support before closing out ticket** - Ask the customer if there is anything else you can do to help them today.",
"chat_history": [
{
"role": "user",
"content": "I would like to return this shirt"
},
{
"role": "assistant",
"content": "Hi there, I'm happy to help with processing this return. Can you please provide an explanation for why you'd like to return this shirt?"
},
{
"role": "user",
"content": "Yes, I am not satisfied with the design"
}
],
"assistant_response": {
"role": "assistant",
"content": "I see. Because the shirt was ordered in the last 30 days, we can provide you with a full refund. Would you like me to process the refund?"
}
}
"""
user_example_2 = """"
Here are the policy instructions:
RETURN POLICY
1. ** Ask the customer why they want the order replaced **
- Categorize their issue into one of the following buckets:
- damaged: They received the product in a damaged state
- satisfaction: The customer is not satisfied with the item and does not like the product.
- unnecessary: They no longer need the item
2a. **If return category is 'damaged'
- Ask customer for a picture of the damaged item
- If the item is indeed damaged, continue to step 3
- If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund
- Skip step 3 and go straight to step 4
2b. **If return category is either 'satisfaction' or 'unnecessary'**
- Ask the customer if they can provide feedback on the quality of the item
- If the order was made within 30 days, notify them that they are eligible for a full refund
- If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%
- If the order was made greater than 60 days ago, notify them that they are not eligible for a refund
3. **If the customer is eligible for a return or refund**
- Ask the customer to confirm that they would like a return or refund
- Once they confirm, process their request
4 **Provide additional support before closing out ticket**
- Ask the customer if there is anything else you can do to help them today.
"""
assistant_example_2 = """
{
"accurate": "false",
"kb_article": "1. ** Ask the customer why they want the order replaced ** - Categorize their issue into one of the following buckets: - damaged: They received the product in a damaged state - satisfaction: The customer is not satisfied with the item and does not like the product. - unnecessary: They no longer need the item 2a. **If return category is 'damaged' - Ask customer for a picture of the damaged item - If the item is indeed damaged, continue to step 3 - If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund - Skip step 3 and go straight to step 4 2b. **If return category is either 'satisfaction' or 'unnecessary'** - Ask the customer if they can provide feedback on the quality of the item - If the order was made within 30 days, notify them that they are eligible for a full refund - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50% - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund 3. **If the customer is eligible for a return or refund** - Ask the customer to confirm that they would like a return or refund - Once they confirm, process their request 4 **Provide additional support before closing out ticket** - Ask the customer if there is anything else you can do to help them today.",
"chat_history": [
{
"role": "user",
"content": "I would like to return this shirt"
},
{
"role": "assistant",
"content": "Hi there, I'm happy to help with processing this return. Can you please provide an explanation for why you'd like to return this shirt?"
},
{
"role": "user",
"content": "Yes, I am not satisfied with the design"
}
],
"assistant_response": {
"role": "assistant",
"content": "I see. Because the shirt was ordered in the last 60 days, we cannot process a refund."
}
}
"""现在让我们遍历策略并生成一些示例。
customer_interactions = []
def fetch_response(policy):
messages = [
{ "role": "system", "content": system_input_prompt},
{ "role": "user", "content": user_example_1},
{ "role": "assistant", "content": assistant_example_1},
{ "role": "user", "content": user_example_2},
{ "role": "assistant", "content": assistant_example_2},
{ "role": "user", "content": policy}
]
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
temperature=0.7,
n=10
)
return response.choices
with ThreadPoolExecutor() as executor:
futures = [executor.submit(fetch_response, policy) for policy in policy_instructions]
for future in futures:
choices = future.result()
customer_interactions.extend([choice.message.content for choice in choices])
interaction_dict = json.loads(customer_interactions[0])
df_interaction = pd.DataFrame([interaction_dict])
# Pretty print the DataFrame
display_dataframe(df_interaction)
| 准确率 | 知识库文章 | 聊天历史 | 助手回复 | |
|---|---|---|---|---|
| 0 | true | PRODUCT FEEDBACK POLICY 1. **Acknowledge Reception** - Thank the customer for taking the time to provide feedback. - Use a personalized greeting: "Thank you for your feedback, [Customer Name]. We appreciate your input." 2. **Categorize Feedback** - Determine the type of feedback: - **Positive Feedback** - **Negative Feedback** - **Suggestions for Improvement** - Document the feedback under the appropriate category in the internal database. 3. **Responding to Positive Feedback** - Express gratitude: "We're thrilled to hear that you enjoyed our product. Thank you for letting us know!" - If possible, offer a small token of appreciation (e.g., discount or voucher for future purchases). 4. **Responding to Negative Feedback** - Apologize sincerely and acknowledge the customer's concerns: "We apologize that our product did not meet your expectations. Your feedback is important to us." - Ask for additional details if necessary to understand the issue better. - Reassure the customer that their feedback will be escalated to the product development team. 5. **Responding to Suggestions** - Acknowledge the suggestion: "Thank you for your suggestion. We value input from our customers as it helps us improve our products." - Inform the customer that their suggestion will be reviewed: "We will share your idea with our product team for further consideration." 6. **Internal Processing** - Log all feedback under the respective category in the internal database. - Forward detailed feedback to the product development team bi-weekly. - High-priority issues should be escalated immediately to the senior management team. 7. **Follow-Up** - Monitor whether the customer's feedback leads to any product updates or changes. - If the customer’s feedback resulted in product enhancement, send a follow-up email to inform them: "Thank you for your valuable feedback. We wanted to let you know that we've made some improvements based on your input." 8. **Closing the Loop** - Ask if there is anything else you can assist the customer with: "Is there anything else we can help you with today?" - Close the ticket once all queries and feedback are appropriately addressed. 9. **Continuous Improvement** - Analyze feedback trends monthly to identify recurring issues and areas for improvement. - Use feedback insights for product development meetings and strategic planning sessions. By following these steps, we ensure that customer feedback is valued, documented, and acted upon to continuously improve our product offerings. | [{'role': 'user', 'content': 'I wanted to let you know that the new app update is fantastic! The interface is so much smoother now.'}] | {'role': 'assistant', 'content': 'Thank you for your feedback! We appreciate your input. We're thrilled to hear that you enjoyed our product. Thank you for letting us know! As a token of our appreciation, we're offering you a 10% discount on your next purchase. Is there anything else we can help you with today?'} |
# Decode the JSON strings
data = [json.loads(entry) for entry in customer_interactions]
# Create a DataFrame from the cleaned data
df = pd.DataFrame(data)df.head(10)| 准确 | 知识库文章 | 聊天历史 | 助手回复 | |
|---|---|---|---|---|
| 0 | true | 产品反馈政策 1. **确认收到**... | [{'role': 'user', 'content': '我想告知...'}] | {'role': 'assistant', 'content': '感谢您...'} |
| 1 | true | 产品反馈政策 1. **确认收到**... | [{'role': 'user', 'content': '我想告诉您... | {'role': 'assistant', 'content': '感谢您... |
| 2 | true | 产品反馈政策 1. **确认收到反馈**... | [{'role': 'user', 'content': '我想提供一些反馈... | {'role': 'assistant', 'content': '感谢您分享您的... |
| 3 | true | 产品反馈政策\n\n1. **确认收悉**... | [{'role': 'user', 'content': '我非常喜欢... | {'role': 'assistant', 'content': '感谢您... |
| 4 | true | 产品反馈政策 1. **确认收到反馈**... | [{'role': 'user', 'content': '我想提供一些... | {'role': 'assistant', 'content': '感谢您... |
| 5 | true | 产品反馈政策 1. **确认收到**... | [{'role': 'user', 'content': '我想告知...'}] | {'role': 'assistant', 'content': '感谢您提...'} |
| 6 | true | 产品反馈政策 1. **确认收到反馈**... | [{'role': 'user', 'content': '我不喜欢这个...'}] | {'role': 'assistant', 'content': '我们为给您...'} |
| 7 | true | 产品反馈政策 1. **确认收到**... | [{'role': 'user', 'content': '我有一些反馈... | {'role': 'assistant', 'content': '感谢您的... |
| 8 | true | 产品反馈政策 1. **确认收到**... | [{'role': 'user', 'content': '我真的很喜欢这个...'}] | {'role': 'assistant', 'content': '感谢您的反馈...'} |
| 9 | true | 1. **确认收到** - 感谢客户... | [{'role': 'user', 'content': '我想说...'}] | {'role': 'assistant', 'content': '感谢您...'} |
2. 构建我们的幻觉防护栏
在构建我们的幻觉防护栏时，以下是一些指导原则：
- 提供非常详细的指标来评估响应是否准确
- 重要的是将"真相"这一概念分解为我们可以衡量的、易于识别的指标
- 诸如真实性和相关性等指标难以衡量。为陈述提供具体的评分方法可以形成更精确的防护栏
- 确保关键术语的一致性
- 保持提示中相关术语的一致性非常重要，例如知识库文章、助手和用户
- 如果我们开始使用诸如助手与智能体这样的短语，模型可能会感到困惑
- 从最先进的模型开始
- 使用最先进的模型时需要在成本和质量之间做出权衡。虽然GPT-4o可能更昂贵，但从最先进的模型开始很重要，这样可以确保高度的准确性
- 在我们全面测试完防护栏并对其性能充满信心后，可以考虑通过将其调整为gpt-3.5-turbo来降低成本
- 独立评估每个句子以及整个回复的整体效果
- 如果智能体返回的响应较长，将其拆分为单句并独立评估可能很有帮助
- 除此之外，整体评估消息的全部意图可以确保您不会丢失重要上下文
考虑到以上所有因素，让我们构建一个防护栏系统并衡量其性能。
guardrail_system_message = """You are a highly specialized assistant tasked with reviewing chatbot responses to identify and flag any inaccuracies or hallucinations. For each user message, you must thoroughly analyze the response by considering:
1. Knowledge Accuracy: Does the message accurately reflect information found in the knowledge base? Assess not only direct mentions but also contextually inferred knowledge.
2. Relevance: Does the message directly address the user's question or statement? Check if the response logically follows the user’s last message, maintaining coherence in the conversation thread.
3. Policy Compliance: Does the message adhere to company policies? Evaluate for subtleties such as misinformation, overpromises, or logical inconsistencies. Ensure the response is polite, non-discriminatory, and practical.
To perform your task you will be given the following:
1. Knowledge Base Articles - These are your source of truth for verifying the content of assistant messages.
2. Chat Transcript - Provides context for the conversation between the user and the assistant.
3. Assistant Message - The message from the assistant that needs review.
For each sentence in the assistant's most recent response, assign a score based on the following criteria:
1. Factual Accuracy:
- Score 1 if the sentence is factually correct and corroborated by the knowledge base.
- Score 0 if the sentence contains factual errors or unsubstantiated claims.
2. Relevance:
- Score 1 if the sentence directly and specifically addresses the user's question or statement without digression.
- Score 0 if the sentence is tangential or does not build logically on the conversation thread.
3. Policy Compliance:
- Score 1 if the response complies with all company policies including accuracy, ethical guidelines, and user engagement standards.
- Score 0 if it violates any aspect of the policies, such as misinformation or inappropriate content.
4. Contextual Coherence:
- Score 1 if the sentence maintains or enhances the coherence of the conversation, connecting logically with preceding messages.
- Score 0 if it disrupts the flow or context of the conversation.
Include in your response an array of JSON objects for each evaluated sentence. Each JSON object should contain:
- `sentence`: Text of the evaluated sentence.
- `factualAccuracy`: Score for factual correctness (0 or 1).
- `factualReference`: If scored 1, cite the exact line(s) from the knowledge base. If scored 0, provide a rationale.
- `relevance`: Score for relevance to the user’s question (0 or 1).
- `policyCompliance`: Score for adherence to company policies (0 or 1).
- `contextualCoherence`: Score for maintaining conversation coherence (0 or 1).
ALWAYS RETURN YOUR RESPONSE AS AN ARRAY OF JSONS.
"""
fs_user_1 = """
## Knowledge Base Articles:
1. ** Ask the customer why they want the order replaced **
- Categorize their issue into one of the following buckets:
- damaged: They received the product in a damaged state
- satisfaction: The customer is not satisfied with the item and does not like the product.
- unnecessary: They no longer need the item
2a. **If return category is 'damaged'
- Ask customer for a picture of the damaged item
- If the item is indeed damaged, continue to step 3
- If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund
- Skip step 3 and go straight to step 4
2b. **If return category is either 'satisfaction' or 'unnecessary'**
- Ask the customer if they can provide feedback on the quality of the item
- If the order was made within 30 days, notify them that they are eligible for a full refund
- If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%
- If the order was made greater than 60 days ago, notify them that they are not eligible for a refund
3. **If the customer is eligible for a return or refund**
- Ask the customer to confirm that they would like a return or refund
- Once they confirm, process their request
4 **Provide additional support before closing out ticket**
- Ask the customer if there is anything else you can do to help them today.
## Chat Transcript:
[
{
"role": "user",
"content: "I would like to return this shirt"
},
{
"role": "assistant",
"content": "Hi there, I'm happy to help with processing this return. Can you please provide an explanation for why you'd like to return this shirt?"
},
{
"role": "user",
"content: "Yes, I am not satisfied with the design"
}
]
## Assistant Message:
I see, because the shirt was ordered in the last 30 days, we can provide you with a full refund. Would you like me to process the refund?
"""
fs_assistant_1 = """[
{
"sentence": "I see, because the shirt was ordered in the last 30 days, we can provide you with a full refund.",
"factualAccuracy": 1,
"factualReference": "If the order was made within 30 days, notify them that they are eligible for a full refund",
"relevance": 1,
"policyCompliance": 1,
"contextualCoherence": 1
},
{
"sentence": "Would you like me to process the refund?",
"factualAccuracy": 1,
"factualReference": "If the order was made within 30 days, notify them that they are eligible for a full refund",
"relevance": 1,
"policyCompliance": 1,
"contextualCoherence": 1
}
]
"""
fs_user_2 = """
## Knowledge Base Articles:
1. ** Ask the customer why they want the order replaced **
- Categorize their issue into one of the following buckets:
- damaged: They received the product in a damaged state
- satisfaction: The customer is not satisfied with the item and does not like the product.
- unnecessary: They no longer need the item
2a. **If return category is 'damaged'
- Ask customer for a picture of the damaged item
- If the item is indeed damaged, continue to step 3
- If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund
- Skip step 3 and go straight to step 4
2b. **If return category is either 'satisfaction' or 'unnecessary'**
- Ask the customer if they can provide feedback on the quality of the item
- If the order was made within 30 days, notify them that they are eligible for a full refund
- If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%
- If the order was made greater than 60 days ago, notify them that they are not eligible for a refund
3. **If the customer is eligible for a return or refund**
- Ask the customer to confirm that they would like a return or refund
- Once they confirm, process their request
4 **Provide additional support before closing out ticket**
- Ask the customer if there is anything else you can do to help them today.
## Chat Transcript:
[
{
"role": "user",
"content: "I would like to return this shirt"
},
{
"role": "assistant",
"content": "Hi there, I'm happy to help with processing this return. Can you please provide an explanation for why you'd like to return this shirt?"
},
{
"role": "user",
"content: "Yes, I am not satisfied with the design"
},
{
"role": "assistant",
"content": "I see, because the shirt was ordered in the last 60 days, we cannot process a refund."
}
]
## Assistant Message:
I see, because the shirt was ordered in the last 60 days, we cannot process a refund.
"""
fs_assistant_2 = """'[
{
"sentence": "I see, because the shirt was ordered in the last 60 days, we cannot process a refund.",
"factualAccuracy": 0,
"knowledgeReference: "If an order was placed within 60 days, you must process a partial refund."
"relevance": 1,
"policyCompliance": 1,
"contextualCoherence": 1
}
]"""
user_input = """
## Knowledge Base Articles
{kb_articles}
## Chat Transcript
{transcript}
## Assistant Message:
{message}
"""hallucination_outputs = []
def validate_hallucinations(row):
kb_articles = row['kb_article']
chat_history = row['chat_history']
assistant_response = row['assistant_response']
user_input_filled = user_input.format(
kb_articles=kb_articles,
transcript=chat_history,
message=assistant_response
)
messages = [
{ "role": "system", "content": guardrail_system_message},
{ "role": "user", "content": fs_user_1},
{ "role": "assistant", "content": fs_assistant_1},
{ "role": "user", "content": fs_user_2},
{ "role": "assistant", "content": fs_assistant_2},
{ "role": "user", "content": user_input_filled}
]
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
temperature=0.7,
n=10
)
return response.choices
# Create an empty list to store the results
results_list = []
def process_row(row):
choices = validate_hallucinations(row)
response_json = choices[0].message.content
# Parse the response content as JSON
response_data = json.loads(response_json)
for response_item in response_data:
# Sum up the scores of the properties
score_sum = (
response_item.get('factualAccuracy', 0) +
response_item.get('relevance', 0) +
response_item.get('policyCompliance', 0) +
response_item.get('contextualCoherence', 0)
)
# Determine if the response item is a pass or fail
hallucination_status = 'Pass' if score_sum == 4 else 'Fail'
results_list.append({
'accurate': row['accurate'],
'hallucination': hallucination_status,
'kb_article': row['kb_article'],
'chat_history': row['chat_history'],
'assistant_response': row['assistant_response']
})
# Use ThreadPoolExecutor to parallelize the processing of rows
with ThreadPoolExecutor() as executor:
executor.map(process_row, [row for index, row in df.iterrows()])
# Convert the list to a DataFrame
results_df = pd.DataFrame(results_list)results_df.head()| 准确 | 幻觉 | 知识库文章 | 聊天历史 | 助手回复 | |
|---|---|---|---|---|---|
| 0 | true | 通过 | 产品反馈政策 1. **确认收到**... | [{'role': 'user', 'content': '我想告诉你...'}] | {'role': 'assistant', 'content': '感谢您提...'} |
| 1 | true | 通过 | 产品反馈政策 1. **确认收到**... | [{'role': 'user', 'content': '我想告知...'}] | {'role': 'assistant', 'content': '感谢您...'} |
| 2 | true | 通过 | 产品反馈政策 1. **确认收到**... | [{'role': 'user', 'content': '我想告知...'}] | {'role': 'assistant', 'content': '感谢您提...'} |
| 3 | true | 通过 | 1. **确认收到** - 感谢客户... | [{'role': 'user', 'content': 'I wanted to say ... | {'role': 'assistant', 'content': 'Thank you fo... |
| 4 | true | 通过 | 1. **确认收到** - 感谢客户... | [{'role': 'user', 'content': '我想说...'}] | {'role': 'assistant', 'content': '感谢您...'} |
results_df.to_csv('hallucination_results.csv', index=False)
df = pd.read_csv('hallucination_results.csv')
if 'accurate' not in df.columns or 'hallucination' not in df.columns:
print("Error: The required columns are not present in the DataFrame.")
else:
# Transform values to binary 0/1
try:
df['accurate'] = df['accurate'].astype(str).str.strip().map(lambda x: 1 if x in ['True', 'true'] else 0)
df['hallucination'] = df['hallucination'].str.strip().map(lambda x: 1 if x == 'Pass' else 0)
except KeyError as e:
print(f"Mapping error: {e}")
# Check for any NaN values after mapping
if df['accurate'].isnull().any() or df['hallucination'].isnull().any():
print("Error: There are NaN values in the mapped columns. Check the input data for unexpected values.")
else:
# Calculate precision and recall
try:
# Precision measures the proportion of correctly identified true positives out of all instances predicted as positive.
# Precision = (True Positives) / (True Positives + False Positives)
precision = precision_score(df['accurate'], df['hallucination'])
# Recall measures the proportion of correctly identified true positives out of all actual positive instances in the dataset.
# Recall = (True Positives) / (True Positives + False Negatives)
recall = recall_score(df['accurate'], df['hallucination'])
print(f"\nPrecision: {precision:.2f} (Precision measures the proportion of correctly identified true positives out of all instances predicted as positive.), "
f"\nRecall: {recall:.2f} (Recall measures the proportion of correctly identified true positives out of all actual positive instances in the dataset.)")
except ValueError as e:
print(f"Error in calculating precision and recall: {e}")Precision: 0.97 (Precision measures the proportion of correctly identified true positives out of all instances predicted as positive.), Recall: 1.00 (Recall measures the proportion of correctly identified true positives out of all actual positive instances in the dataset.)
从上述结果可以看出，该程序表现良好，具有较高的精确率和召回率指标。这意味着防护机制能够准确识别模型输出中的幻觉现象。