使用TransformMessages预处理聊天记录
介绍
本笔记本展示了如何使用TransformMessages赋予任何ConversableAgent处理长上下文、敏感数据等的能力。
Requirements
import copy
import pprint
import re
from typing import Dict, List, Tuple
import autogen
from autogen.agentchat.contrib.capabilities import transform_messages, transforms
config_list = autogen.config_list_from_json(
env_or_file="OAI_CONFIG_LIST",
)
# Define your llm config
llm_config = {"config_list": config_list}
tip
了解更多关于为agent配置LLM的信息在这里.
# Define your agent; the user proxy and an assistant
assistant = autogen.AssistantAgent(
"assistant",
llm_config=llm_config,
)
user_proxy = autogen.UserProxyAgent(
"user_proxy",
human_input_mode="NEVER",
is_termination_msg=lambda x: "TERMINATE" in x.get("content", ""),
max_consecutive_auto_reply=10,
)
处理长上下文
想象一个场景，LLM生成了大量文本，超出了API提供者设置的令牌限制。为了解决这个问题，你可以利用TransformMessages以及其组成的转换功能，MessageHistoryLimiter和MessageTokenLimiter。
MessageHistoryLimiter: 您可以限制作为上下文历史记录的消息总数。当您想将会话上下文限制为特定数量的最新消息时，这种转换特别有用，确保高效的处理和响应生成。MessageTokenLimiter: 允许您限制总token数，可以是每条消息的token数或整个上下文历史中的token数（或两者）。当您需要遵守API提供商规定的严格token限制时，此转换非常有用，可以防止因超过允许的token数而产生不必要的成本或错误。此外，还可以应用min_tokens阈值，确保仅在token数不少于指定阈值时才应用此转换。
# Limit the message history to the 3 most recent messages
max_msg_transfrom = transforms.MessageHistoryLimiter(max_messages=3)
# Limit the token limit per message to 10 tokens
token_limit_transform = transforms.MessageTokenLimiter(max_tokens_per_message=3, min_tokens=10)
示例1：限制消息数量
让我们来看一下这些转换将如何影响消息。
下面我们看到通过应用MessageHistoryLimiter，我们可以看到
将上下文历史限制为最近的3条消息。
messages = [
{"role": "user", "content": "hello"},
{"role": "assistant", "content": [{"type": "text", "text": "there"}]},
{"role": "user", "content": "how"},
{"role": "assistant", "content": [{"type": "text", "text": "are you doing?"}]},
{"role": "user", "content": "very very very very very very long string"},
]
processed_messages = max_msg_transfrom.apply_transform(copy.deepcopy(messages))
pprint.pprint(processed_messages)
[{'content': 'how', 'role': 'user'},
{'content': [{'text': 'are you doing?', 'type': 'text'}], 'role': 'assistant'},
{'content': 'very very very very very very long string', 'role': 'user'}]
示例 2：限制令牌数量
现在让我们测试限制消息中的标记数量。我们可以看到可以将标记数量限制为3，在这种情况下相当于3个单词。
processed_messages = token_limit_transform.apply_transform(copy.deepcopy(messages))
pprint.pprint(processed_messages)
[{'content': 'hello', 'role': 'user'},
{'content': [{'text': 'there', 'type': 'text'}], 'role': 'assistant'},
{'content': 'how', 'role': 'user'},
{'content': [{'text': 'are you doing', 'type': 'text'}], 'role': 'assistant'},
{'content': 'very very very', 'role': 'user'}]
此外，min_tokens阈值被设置为10，这意味着如果消息中的总令牌数小于该值，则不会应用转换。这在转换应仅在达到一定数量的令牌后才发生时特别有用，例如在模型的上下文窗口中。下面提供了一个示例。
short_messages = [
{"role": "user", "content": "hello there, how are you?"},
{"role": "assistant", "content": [{"type": "text", "text": "hello"}]},
]
processed_short_messages = token_limit_transform.apply_transform(copy.deepcopy(short_messages))
pprint.pprint(processed_short_messages)
[{'content': 'hello there, how are you?', 'role': 'user'},
{'content': [{'text': 'hello', 'type': 'text'}], 'role': 'assistant'}]
示例3：结合转换
让我们用代理来测试这些转换（即将进行的测试是从agentchat_capability_long_context_handling笔记本中复制的）。我们将会看到，没有处理长上下文能力的代理会导致错误，而具备该能力的代理则不会有问题。
assistant_base = autogen.AssistantAgent(
"assistant",
llm_config=llm_config,
)
assistant_with_context_handling = autogen.AssistantAgent(
"assistant",
llm_config=llm_config,
)
# suppose this capability is not available
context_handling = transform_messages.TransformMessages(
transforms=[
transforms.MessageHistoryLimiter(max_messages=10),
transforms.MessageTokenLimiter(max_tokens=1000, max_tokens_per_message=50, min_tokens=500),
]
)
context_handling.add_to_agent(assistant_with_context_handling)
user_proxy = autogen.UserProxyAgent(
"user_proxy",
human_input_mode="NEVER",
is_termination_msg=lambda x: "TERMINATE" in x.get("content", ""),
code_execution_config={
"work_dir": "coding",
"use_docker": False,
},
max_consecutive_auto_reply=2,
)
# suppose the chat history is large
# Create a very long chat history that is bound to cause a crash
# for gpt 3.5
for i in range(1000):
# define a fake, very long messages
assitant_msg = {"role": "assistant", "content": "test " * 1000}
user_msg = {"role": "user", "content": ""}
assistant_base.send(assitant_msg, user_proxy, request_reply=False, silent=True)
assistant_with_context_handling.send(assitant_msg, user_proxy, request_reply=False, silent=True)
user_proxy.send(user_msg, assistant_base, request_reply=False, silent=True)
user_proxy.send(user_msg, assistant_with_context_handling, request_reply=False, silent=True)
try:
user_proxy.initiate_chat(assistant_base, message="plot and save a graph of x^2 from -10 to 10", clear_history=False)
except Exception as e:
print("Encountered an error with the base assistant")
print(e)
print("\n\n")
try:
user_proxy.initiate_chat(
assistant_with_context_handling, message="plot and save a graph of x^2 from -10 to 10", clear_history=False
)
except Exception as e:
print(e)
user_proxy (to assistant):
plot and save a graph of x^2 from -10 to 10
--------------------------------------------------------------------------------
Encountered an error with the base assistant
Error code: 400 - {'error': {'message': "This model's maximum context length is 16385 tokens. However, your messages resulted in 1009487 tokens. Please reduce the length of the messages.", 'type': 'invalid_request_error', 'param': 'messages', 'code': 'context_length_exceeded'}}
user_proxy (to assistant):
plot and save a graph of x^2 from -10 to 10
--------------------------------------------------------------------------------
Removed 1991 messages. Number of messages reduced from 2001 to 10.
Truncated 3804 tokens. Number of tokens reduced from 4019 to 215
assistant (to user_proxy):
```python
# filename: plot_x_squared.py
import matplotlib.pyplot as plt
import numpy as np
# Generate an array of x values from -10 to 10
x = np.linspace(-10, 10, 400)
# Calculate the y values by squaring the x values
y = x**2
# Create the plot
plt.figure()
plt.plot(x, y)
# Title and labels
plt.title('Graph of y = x^2')
plt.xlabel('x')
plt.ylabel('y')
# Save the plot as a file
plt.savefig('x_squared_plot.png')
# Show the plot
plt.show()
```
Please save the above code into a file named `plot_x_squared.py`. After saving the code, you can execute it to generate and save the graph of y = x^2 from -10 to 10. The graph will also be displayed to you and the file `x_squared_plot.png` will be created in the current directory. Make sure you have `matplotlib` and `numpy` libraries installed in your Python environment before executing the code. If they are not installed, you can install them using `pip`:
```sh
pip install matplotlib numpy
```
--------------------------------------------------------------------------------
>>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...
>>>>>>>> EXECUTING CODE BLOCK 1 (inferred language is sh)...
user_proxy (to assistant):
exitcode: 0 (execution succeeded)
Code output:
Figure(640x480)
Requirement already satisfied: matplotlib in c:\users\bt314mc\appdata\local\programs\python\python311\lib\site-packages (3.8.0)
Requirement already satisfied: numpy in c:\users\bt314mc\appdata\local\programs\python\python311\lib\site-packages (1.26.0)
Requirement already satisfied: contourpy>=1.0.1 in c:\users\bt314mc\appdata\local\programs\python\python311\lib\site-packages (from matplotlib) (1.1.1)
Requirement already satisfied: cycler>=0.10 in c:\users\bt314mc\appdata\local\programs\python\python311\lib\site-packages (from matplotlib) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in c:\users\bt314mc\appdata\local\programs\python\python311\lib\site-packages (from matplotlib) (4.42.1)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\users\bt314mc\appdata\local\programs\python\python311\lib\site-packages (from matplotlib) (1.4.5)
Requirement already satisfied: packaging>=20.0 in c:\users\bt314mc\appdata\local\programs\python\python311\lib\site-packages (from matplotlib) (23.2)
Requirement already satisfied: pillow>=6.2.0 in c:\users\bt314mc\appdata\local\programs\python\python311\lib\site-packages (from matplotlib) (10.0.1)
Requirement already satisfied: pyparsing>=2.3.1 in c:\users\bt314mc\appdata\local\programs\python\python311\lib\site-packages (from matplotlib) (3.1.1)
Requirement already satisfied: python-dateutil>=2.7 in c:\users\bt314mc\appdata\local\programs\python\python311\lib\site-packages (from matplotlib) (2.8.2)
Requirement already satisfied: six>=1.5 in c:\users\bt314mc\appdata\local\programs\python\python311\lib\site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)
--------------------------------------------------------------------------------
Removed 1993 messages. Number of messages reduced from 2003 to 10.
Truncated 3523 tokens. Number of tokens reduced from 3788 to 265
assistant (to user_proxy):
It appears that the matplotlib library is already installed on your system, and the previous script started successfully but did not finish because the plotting code was incomplete.
I will provide you with the full code to plot and save the graph of \( x^2 \) from -10 to 10.
```python
# filename: plot_x_squared.py
import matplotlib.pyplot as plt
import numpy as np
# Generate an array of x values from -10 to 10
x = np.linspace(-10, 10, 400)
# Calculate the y values based on the x values
y = x**2
# Create the plot
plt.figure(figsize=(8, 6))
plt.plot(x, y, label='y = x^2')
# Add a title and labels
plt.title('Plot of y = x^2')
plt.xlabel('x')
plt.ylabel('y')
# Add a legend
plt.legend()
# Save the figure
plt.savefig('plot_x_squared.png')
# Show the plot
plt.show()
```
Please execute this Python code in its entirety. It will create a graph of \( y = x^2 \) with x values ranging from -10 to 10, and then it will save the graph as a PNG file named 'plot_x_squared.png' in the current working directory. It will also display the plot window with the graph.
--------------------------------------------------------------------------------
>>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...
user_proxy (to assistant):
exitcode: 0 (execution succeeded)
Code output:
Figure(800x600)
--------------------------------------------------------------------------------
Removed 1995 messages. Number of messages reduced from 2005 to 10.
Truncated 2802 tokens. Number of tokens reduced from 3086 to 284
assistant (to user_proxy):
It seems the graph has been generated, but the output doesn't tell us if the graph was saved. The expected behavior was to have a file saved in the current working directory. Can you please check in your current directory for a file named `plot_x_squared.png`? If it exists, then the task is complete.
If you don't find the file, let me know, and I will troubleshoot further.
--------------------------------------------------------------------------------
处理敏感数据
你可以使用MessageTransform协议来创建自定义的消息转换，以从聊天历史中删除敏感数据。这在你想确保敏感信息（如API密钥、密码或个人数据）不会在聊天历史或日志中暴露时特别有用。
现在，我们将创建一个自定义消息转换器来检测任何OpenAI API密钥并对其进行编辑。
# The transform must adhere to transform_messages.MessageTransform protocol.
class MessageRedact:
def __init__(self):
self._openai_key_pattern = r"sk-([a-zA-Z0-9]{48})"
self._replacement_string = "REDACTED"
def apply_transform(self, messages: List[Dict]) -> List[Dict]:
temp_messages = copy.deepcopy(messages)
for message in temp_messages:
if isinstance(message["content"], str):
message["content"] = re.sub(self._openai_key_pattern, self._replacement_string, message["content"])
elif isinstance(message["content"], list):
for item in message["content"]:
if item["type"] == "text":
item["text"] = re.sub(self._openai_key_pattern, self._replacement_string, item["text"])
return temp_messages
def get_logs(self, pre_transform_messages: List[Dict], post_transform_messages: List[Dict]) -> Tuple[str, bool]:
keys_redacted = self._count_redacted(post_transform_messages) - self._count_redacted(pre_transform_messages)
if keys_redacted > 0:
return f"Redacted {keys_redacted} OpenAI API keys.", True
return "", False
def _count_redacted(self, messages: List[Dict]) -> int:
# counts occurrences of "REDACTED" in message content
count = 0
for message in messages:
if isinstance(message["content"], str):
if "REDACTED" in message["content"]:
count += 1
elif isinstance(message["content"], list):
for item in message["content"]:
if isinstance(item, dict) and "text" in item:
if "REDACTED" in item["text"]:
count += 1
return count
assistant_with_redact = autogen.AssistantAgent(
"assistant",
llm_config=llm_config,
max_consecutive_auto_reply=1,
)
# suppose this capability is not available
redact_handling = transform_messages.TransformMessages(transforms=[MessageRedact()])
redact_handling.add_to_agent(assistant_with_redact)
user_proxy = autogen.UserProxyAgent(
"user_proxy",
human_input_mode="NEVER",
max_consecutive_auto_reply=1,
)
messages = [
{"content": "api key 1 = sk-7nwt00xv6fuegfu3gnwmhrgxvuc1cyrhxcq1quur9zvf05fy"}, # Don't worry, randomly generated
{"content": [{"type": "text", "text": "API key 2 = sk-9wi0gf1j2rz6utaqd3ww3o6c1h1n28wviypk7bd81wlj95an"}]},
]
for message in messages:
user_proxy.send(message, assistant_with_redact, request_reply=False, silent=True)
result = user_proxy.initiate_chat(
assistant_with_redact, message="What are the two API keys that I just provided", clear_history=False
)
user_proxy (to assistant):
What are the two API keys that I just provided
--------------------------------------------------------------------------------
Redacted 2 OpenAI API keys.
assistant (to user_proxy):
As an AI, I must inform you that it is not safe to share API keys publicly as they can be used to access your private data or services that can incur costs. Given that you've typed "REDACTED" instead of the actual keys, it seems you are aware of the privacy concerns and are likely testing my response or simulating an exchange without exposing real credentials, which is a good practice for privacy and security reasons.
To respond directly to your direct question: The two API keys you provided are both placeholders indicated by the text "REDACTED", and not actual API keys. If these were real keys, I would have reiterated the importance of keeping them secure and would not display them here.
Remember to keep your actual API keys confidential to prevent unauthorized use. If you've accidentally exposed real API keys, you should revoke or regenerate them as soon as possible through the corresponding service's API management console.
--------------------------------------------------------------------------------
user_proxy (to assistant):
--------------------------------------------------------------------------------
Redacted 2 OpenAI API keys.