Skip to main content

Assistants API

涵盖了线程、消息和助手。

LiteLLM 目前支持：

  • 创建助手
  • 删除助手
  • 获取助手
  • 创建线程
  • 获取线程
  • 添加消息
  • 获取消息
  • 运行线程

支持的提供商：​

快速入门​

调用一个现有的助手。

  • 获取助手

  • 当用户开始对话时创建一个线程。

  • 随着用户提问，将消息添加到线程中。

  • 在调用模型和工具的基础上，运行助手在线程上生成响应。

SDK + 代理​

创建一个助手

import litellm
import os 

# 设置环境变量
os.environ["OPENAI_API_KEY"] = "sk-.."

assistant = litellm.create_assistants(
            custom_llm_provider="openai",
            model="gpt-4-turbo",
            instructions="你是一个私人数学导师。当被问到一个问题时，编写并运行 Python 代码来回答这个问题。",
            name="数学导师",
            tools=[{"type": "code_interpreter"}],
)

### 异步使用 ### 
# assistant = await litellm.acreate_assistants(
#             custom_llm_provider="openai",
#             model="gpt-4-turbo",
#             instructions="你是一个私人数学导师。当被问到一个问题时，编写并运行 Python 代码来回答这个问题。",
#             name="数学导师",
#             tools=[{"type": "code_interpreter"}],
# )

获取助手

from litellm import get_assistants, aget_assistants
import os 

# 设置环境变量
os.environ["OPENAI_API_KEY"] = "sk-.."

assistants = get_assistants(custom_llm_provider="openai")

### 异步使用 ### 
# assistants = await aget_assistants(custom_llm_provider="openai")

创建一个线程

from litellm import create_thread, acreate_thread
import os 

os.environ["OPENAI_API_KEY"] = "sk-.."

new_thread = create_thread(
            custom_llm_provider="openai",
            messages=[{"role": "user", "content": "嘿，最近怎么样？"}],  # type: ignore
        )

### 异步使用 ### 
# new_thread = await acreate_thread(custom_llm_provider="openai",messages=[{"role": "user", "content": "嘿，最近怎么样？"}])

向线程添加消息

from litellm import create_thread, get_thread, aget_thread, add_message, a_add_message
import os 

os.environ["OPENAI_API_KEY"] = "sk-.."

## 创建一个线程
_new_thread = create_thread(
            custom_llm_provider="openai",
            messages=[{"role": "user", "content": "嘿，最近怎么样？"}],  # type: ignore
        )

## 或者获取现有的线程
received_thread = get_thread(
            custom_llm_provider="openai",
            thread_id=_new_thread.id,
        )

### 异步使用 ### 
# received_thread = await aget_thread(custom_llm_provider="openai", thread_id=_new_thread.id,)

## 向线程添加消息
message = {"role": "user", "content": "嘿，最近怎么样？"}
added_message = add_message(
            thread_id=_new_thread.id, custom_llm_provider="openai", **message
        )

### 异步使用 ### 
# added_message = await a_add_message(thread_id=_new_thread.id, custom_llm_provider="openai", **message)

在线程上运行助手

from litellm import get_assistants, create_thread, add_message, run_thread, arun_thread
import os 

os.environ["OPENAI_API_KEY"] = "sk-.."
assistants = get_assistants(custom_llm_provider="openai")

## 获取第一个助手 ###
assistant_id = assistants.data[0].id

## 获取一个线程
_new_thread = create_thread(
            custom_llm_provider="openai",
            messages=[{"role": "user", "content": "嘿，最近怎么样？"}],  # type: ignore
        )

## 添加消息
message = {"role": "user", "content": "嘿，最近怎么样？"}
added_message = add_message(
            thread_id=_new_thread.id, custom_llm_provider="openai", **message
        )

## 🚨 运行线程
response = run_thread(
            custom_llm_provider="openai", thread_id=thread_id, assistant_id=assistant_id
        )

### 异步使用 ### 
# response = await arun_thread(custom_llm_provider="openai", thread_id=thread_id, assistant_id=assistant_id)

print(f"run_thread: {run_thread}")
assistant_settings:
  custom_llm_provider: azure
  litellm_params: 
    api_key: os.environ/AZURE_API_KEY
    api_base: os.environ/AZURE_API_BASE
    api_version: os.environ/AZURE_API_VERSION
$ litellm --config /path/to/config.yaml

# 运行在 http://0.0.0.0:4000

创建助手

curl "http://localhost:4000/v1/assistants" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-1234" \
  -d '{
    "instructions": "你是一位私人数学导师。当被问到问题时，编写并运行Python代码来回答问题。",
    "name": "数学导师",
    "tools": [{"type": "代码解释器"}],
    "model": "gpt-4-turbo"
  }'

获取助手

curl "http://0.0.0.0:4000/v1/assistants?order=desc&limit=20" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-1234"

创建线程

curl http://0.0.0.0:4000/v1/threads \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-1234" \
  -d ''

获取线程

curl http://0.0.0.0:4000/v1/threads/{thread_id} \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-1234"

向线程添加消息

curl http://0.0.0.0:4000/v1/threads/{thread_id}/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-1234" \
  -d '{
      "role": "user",
      "content": "AI是如何工作的？用简单的术语解释一下。"
    }'

在线程上运行助手

curl http://0.0.0.0:4000/v1/threads/thread_abc123/runs \
  -H "Authorization: Bearer sk-1234" \
  -H "Content-Type: application/json" \
  -d '{
    "assistant_id": "asst_abc123"
  }'

流式传输​

from litellm import run_thread_stream 
import os

os.environ["OPENAI_API_KEY"] = "sk-.."

message = {"role": "user", "content": "嘿，最近怎么样？"}  

data = {"custom_llm_provider": "openai", "thread_id": _new_thread.id, "assistant_id": assistant_id, **message}

run = run_thread_stream(**data)
with run as run:
    assert isinstance(run, AssistantEventHandler)
    for chunk in run: 
      print(f"chunk: {chunk}")
    run.until_done()
curl -X POST 'http://0.0.0.0:4000/threads/{thread_id}/runs' \
-H 'Authorization: Bearer sk-1234' \
-H 'Content-Type: application/json' \
-D '{
      "assistant_id": "asst_6xVZQFFy1Kw87NbnYeNebxTf",
      "stream": true
}'

👉 代理API参考​

Azure OpenAI​

配置

assistant_settings:
  custom_llm_provider: azure
  litellm_params: 
    api_key: os.environ/AZURE_API_KEY
    api_base: os.environ/AZURE_API_BASE

curl

curl -X POST "http://localhost:4000/v1/assistants" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-1234" \
  -d '{
    "instructions": "你是一位个人数学导师。当被问到问题时，编写并运行Python代码来回答问题。",
    "name": "数学导师",
    "tools": [{"type": "code_interpreter"}],
    "model": "<我的Azure部署名称>"
  }'

OpenAI兼容的API​

要调用OpenAI兼容的助手API（例如Astra助手API），只需在模型名称前添加openai/：

配置

assistant_settings:
  custom_llm_provider: openai
  litellm_params: 
    api_key: os.environ/ASTRA_API_KEY
    api_base: os.environ/ASTRA_API_BASE

curl

curl -X POST "http://localhost:4000/v1/assistants" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-1234" \
  -d '{
    "instructions": "你是一位个人数学导师。当被问到问题时，编写并运行Python代码来回答问题。",
    "name": "数学导师",
    "tools": [{"type": "code_interpreter"}],
    "model": "openai/<我的Astra模型名称>"
  }'
优云智算