Cohere
Cohere 是一个基于云端的平台,提供其自有的LLMs,特别是Command系列模型。
Cohere的API与OpenAI的不同,OpenAI是AutoGen使用的原生API,因此要使用Cohere的LLMs,你需要使用这个库。
你需要一个Cohere账户并创建一个API密钥。详情请见他们的网站。
功能
使用此客户端类时,AutoGen的消息会自动 调整以适应Cohere API的特定要求。
此外,该客户端类还支持函数/工具调用,并将根据Cohere的API费用(截至2024年7月)正确跟踪token使用情况和成本。
入门
首先你需要安装autogen-agentchat~=0.2
包,以便与Cohere API库一起使用AutoGen。
pip install autogen-agentchat[cohere]~=0.2
Cohere 提供了许多可使用的模型,如下所示。查看模型列表。
请查看下面的示例 OAI_CONFIG_LIST
,它展示了如何通过将 api_type
指定为 cohere
来使用 Cohere 客户端类。
[
{
"model": "gpt-35-turbo",
"api_key": "your OpenAI Key goes here",
},
{
"model": "gpt-4-vision-preview",
"api_key": "your OpenAI Key goes here",
},
{
"model": "dalle",
"api_key": "your OpenAI Key goes here",
},
{
"model": "command-r-plus",
"api_key": "your Cohere API Key goes here",
"api_type": "cohere"
},
{
"model": "command-r",
"api_key": "your Cohere API Key goes here",
"api_type": "cohere"
},
{
"model": "command",
"api_key": "your Cohere API Key goes here",
"api_type": "cohere"
}
]
作为配置中api_key
键和值的替代方案,你可以将环境变量COHERE_API_KEY
设置为你的Cohere密钥。
export COHERE_API_KEY="your_cohere_api_key_here"
Windows:
set COHERE_API_KEY=your_cohere_api_key_here
API参数
以下参数可以添加到Cohere API的配置中。请参阅此链接以获取更多信息及其默认值。
- temperature(数字 > 0)
- p (数字 0.01..0.99)
- k (数值范围 0..500)
- max_tokens (null, 整数 >= 0)
- seed(空值,整数)
- frequency_penalty (数字 0..1)
- presence_penalty (数字 0..1)
- client_name (null, string)
示例:
[
{
"model": "command-r",
"api_key": "your Cohere API Key goes here",
"api_type": "cohere",
"client_name": "autogen-cohere",
"temperature": 0.5,
"p": 0.2,
"k": 100,
"max_tokens": 2048,
"seed": 42,
"frequency_penalty": 0.5,
"presence_penalty": 0.2
}
]
双代理编码示例
在这个示例中,我们运行了一个双代理聊天,其中包含一个AssistantAgent(主要是一个编码代理)来生成代码,用于计算1到10,000之间的质数数量,然后执行该代码。
我们将使用Cohere的Command R模型,该模型适用于编码。
import os
config_list = [
{
# Let's choose the Command-R model
"model": "command-r",
# Provide your Cohere's API key here or put it into the COHERE_API_KEY environment variable.
"api_key": os.environ.get("COHERE_API_KEY"),
# We specify the API Type as 'cohere' so it uses the Cohere client class
"api_type": "cohere",
}
]
重要的是,我们已经调整了系统消息,以便模型不会返回终止关键字,我们已将其更改为 FINISH,与代码块一起。
from pathlib import Path
from autogen import AssistantAgent, UserProxyAgent
from autogen.coding import LocalCommandLineCodeExecutor
# Setting up the code executor
workdir = Path("coding")
workdir.mkdir(exist_ok=True)
code_executor = LocalCommandLineCodeExecutor(work_dir=workdir)
# Setting up the agents
# The UserProxyAgent will execute the code that the AssistantAgent provides
user_proxy_agent = UserProxyAgent(
name="User",
code_execution_config={"executor": code_executor},
is_termination_msg=lambda msg: "FINISH" in msg.get("content"),
)
system_message = """You are a helpful AI assistant who writes code and the user executes it.
Solve tasks using your coding and language skills.
In the following cases, suggest python code (in a python coding block) for the user to execute.
Solve the task step by step if you need to. If a plan is not provided, explain your plan first. Be clear which step uses code, and which step uses your language skill.
When using code, you must indicate the script type in the code block. The user cannot provide any other feedback or perform any other action beyond executing the code you suggest. The user can't modify your code. So do not suggest incomplete code which requires users to modify. Don't use a code block if it's not intended to be executed by the user.
Don't include multiple code blocks in one response. Do not ask users to copy and paste the result. Instead, use 'print' function for the output when relevant. Check the execution result returned by the user.
If the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes. If the error can't be fixed or if the task is not solved even after the code is executed successfully, analyze the problem, revisit your assumption, collect additional info you need, and think of a different approach to try.
When you find an answer, verify the answer carefully. Include verifiable evidence in your response if possible.
IMPORTANT: Wait for the user to execute your code and then you can reply with the word "FINISH". DO NOT OUTPUT "FINISH" after your code block."""
# The AssistantAgent, using Cohere's model, will take the coding request and return code
assistant_agent = AssistantAgent(
name="Cohere Assistant",
system_message=system_message,
llm_config={"config_list": config_list},
)
/usr/local/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
# Start the chat, with the UserProxyAgent asking the AssistantAgent the message
chat_result = user_proxy_agent.initiate_chat(
assistant_agent,
message="Provide code to count the number of prime numbers from 1 to 10000.",
)
User (to Cohere Assistant):
Provide code to count the number of prime numbers from 1 to 10000.
--------------------------------------------------------------------------------
Cohere
Here's the code to count the number of prime numbers from 1 to 10,000:
```python
# Prime Number Counter
count = 0
for num in range(2, 10001):
if num > 1:
for div in range(2, num):
if (num % div) == 0:
break
else:
count += 1
print(count)
```
My plan is to use two nested loops. The outer loop iterates through numbers from 2 to 10,000. The inner loop checks if there's any divisor for the current number in the range from 2 to the number itself. If there's no such divisor, the number is prime and the counter is incremented.
Please execute the code and let me know the output.
--------------------------------------------------------------------------------
>>>>>>>> NO HUMAN INPUT RECEIVED.
>>>>>>>> USING AUTO REPLY...
>>>>>>>> EXECUTING CODE BLOCK (inferred language is python)...
User (to Cohere Assistant):
exitcode: 0 (execution succeeded)
Code output: 1229
--------------------------------------------------------------------------------
Cohere
That's correct! The code you executed successfully found 1229 prime numbers within the specified range.
FINISH.
--------------------------------------------------------------------------------
>>>>>>>> NO HUMAN INPUT RECEIVED.
工具调用示例
在这个例子中,我们将展示Cohere的Command R+模型如何执行并行工具调用,而不是编写代码,它会建议同时调用多个工具。
我们将使用一个简单的旅行助手程序,其中包含了天气和货币转换的几个工具。
我们首先导入库并设置我们的配置,以便使用Command R+和cohere
客户端类。
import json
import os
from typing import Literal
from typing_extensions import Annotated
import autogen
config_list = [
{"api_type": "cohere", "model": "command-r-plus", "api_key": os.getenv("COHERE_API_KEY"), "cache_seed": None}
]
创建我们的两个代理。
# Create the agent for tool calling
chatbot = autogen.AssistantAgent(
name="chatbot",
system_message="""For currency exchange and weather forecasting tasks,
only use the functions you have been provided with.
Output 'HAVE FUN!' when an answer has been provided.""",
llm_config={"config_list": config_list},
)
# Note that we have changed the termination string to be "HAVE FUN!"
user_proxy = autogen.UserProxyAgent(
name="user_proxy",
is_termination_msg=lambda x: x.get("content", "") and "HAVE FUN!" in x.get("content", ""),
human_input_mode="NEVER",
max_consecutive_auto_reply=1,
)
创建这两个函数,并对它们进行注释,以便这些描述可以传递给LLM。
我们通过使用register_for_execution
函数将用户代理与之关联,以便它可以执行该函数,并使用register_for_llm
函数将聊天机器人(由LLM驱动)与之关联,以便它可以将函数定义传递给LLM。
# Currency Exchange function
CurrencySymbol = Literal["USD", "EUR"]
# Define our function that we expect to call
def exchange_rate(base_currency: CurrencySymbol, quote_currency: CurrencySymbol) -> float:
if base_currency == quote_currency:
return 1.0
elif base_currency == "USD" and quote_currency == "EUR":
return 1 / 1.1
elif base_currency == "EUR" and quote_currency == "USD":
return 1.1
else:
raise ValueError(f"Unknown currencies {base_currency}, {quote_currency}")
# Register the function with the agent
@user_proxy.register_for_execution()
@chatbot.register_for_llm(description="Currency exchange calculator.")
def currency_calculator(
base_amount: Annotated[float, "Amount of currency in base_currency"],
base_currency: Annotated[CurrencySymbol, "Base currency"] = "USD",
quote_currency: Annotated[CurrencySymbol, "Quote currency"] = "EUR",
) -> str:
quote_amount = exchange_rate(base_currency, quote_currency) * base_amount
return f"{format(quote_amount, '.2f')} {quote_currency}"
# Weather function
# Example function to make available to model
def get_current_weather(location, unit="fahrenheit"):
"""Get the weather for some location"""
if "chicago" in location.lower():
return json.dumps({"location": "Chicago", "temperature": "13", "unit": unit})
elif "san francisco" in location.lower():
return json.dumps({"location": "San Francisco", "temperature": "55", "unit": unit})
elif "new york" in location.lower():
return json.dumps({"location": "New York", "temperature": "11", "unit": unit})
else:
return json.dumps({"location": location, "temperature": "unknown"})
# Register the function with the agent
@user_proxy.register_for_execution()
@chatbot.register_for_llm(description="Weather forecast for US cities.")
def weather_forecast(
location: Annotated[str, "City name"],
) -> str:
weather_details = get_current_weather(location=location)
weather = json.loads(weather_details)
return f"{weather['location']} will be {weather['temperature']} degrees {weather['unit']}"
我们传递客户的消息并运行聊天。
最后,我们要求LLM(大语言模型)总结聊天内容并打印出来。
# start the conversation
res = user_proxy.initiate_chat(
chatbot,
message="What's the weather in New York and can you tell me how much is 123.45 EUR in USD so I can spend it on my holiday? Throw a few holiday tips in as well.",
summary_method="reflection_with_llm",
)
print(f"LLM SUMMARY: {res.summary['content']}")
What's the weather in New York and can you tell me how much is 123.45 EUR in USD so I can spend it on my holiday? Throw a few holiday tips in as well.
--------------------------------------------------------------------------------
I will use the weather_forecast function to find out the weather in New York, and the currency_calculator function to convert 123.45 EUR to USD. I will then search for 'holiday tips' to find some extra information to include in my answer.
***** Suggested tool call (45212): weather_forecast *****
Arguments:
{"location": "New York"}
*********************************************************
***** Suggested tool call (16564): currency_calculator *****
Arguments:
{"base_amount": 123.45, "base_currency": "EUR", "quote_currency": "USD"}
************************************************************
--------------------------------------------------------------------------------
>>>>>>>> EXECUTING FUNCTION weather_forecast...
>>>>>>>> EXECUTING FUNCTION currency_calculator...
***** Response from calling tool (45212) *****
New York will be 11 degrees fahrenheit
**********************************************
--------------------------------------------------------------------------------
***** Response from calling tool (16564) *****
135.80 USD
**********************************************
--------------------------------------------------------------------------------
The weather in New York is 11 degrees Fahrenheit.
€123.45 is worth $135.80.
Here are some holiday tips:
- Make sure to pack layers for the cold weather
- Try the local cuisine, New York is famous for its pizza
- Visit Central Park and take in the views from the top of the Rockefeller Centre
HAVE FUN!
--------------------------------------------------------------------------------
LLM SUMMARY: The weather in New York is 11 degrees Fahrenheit. 123.45 EUR is worth 135.80 USD. Holiday tips: make sure to pack warm clothes and have a great time!
我们可以看到,Command R+ 建议我们调用两个工具并传递了正确的参数。 user_proxy
执行了它们,并将结果传递回 Command R+ 进行解释和回应。最后,Command R+ 被要求总结整个对话。