Skip to main content

Braintrust - 评估 + 日志记录

Braintrust 管理 AI 产品的评估、日志记录、提示测试场以及数据管理。

快速开始​

# pip install langfuse 
import litellm
import os

# 设置环境变量
os.environ["BRAINTRUST_API_KEY"] = "" 
os.environ['OPENAI_API_KEY']=""

# 将 braintrust 设置为回调函数，litellm 将数据发送到 braintrust
litellm.callbacks = ["braintrust"] 
 
# openai 调用
response = litellm.completion(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "user", "content": "Hi 👋 - i'm openai"}
  ]
)

OpenAI 代理使用​

  1. 添加密钥到环境变量
BRAINTRUST_API_KEY=""
  1. 将 braintrust 添加到回调函数
model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: gpt-3.5-turbo
      api_key: os.environ/OPENAI_API_KEY

litellm_settings:
  callbacks: ["braintrust"]
  1. 测试它！
curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-D '{
    "model": "groq-llama3",
    "messages": [
        { "role": "system", "content": "Use your tools smartly"},
        { "role": "user", "content": "What time is it now? Use your tool"}
    ]
}'

高级用法 - 传递项目 ID​

response = litellm.completion(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "user", "content": "Hi 👋 - i'm openai"}
  ], 
  metadata={
    "project_id": "my-special-project" 
  }
)

Curl

curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-D '{
    "model": "groq-llama3",
    "messages": [
        { "role": "system", "content": "Use your tools smartly"},
        { "role": "user", "content": "What time is it now? Use your tool"}
    ],
    "metadata": {
        "project_id": "my-special-project"
    }
}'

OpenAI SDK

import openai
client = openai.OpenAI(
    api_key="anything",
    base_url="http://0.0.0.0:4000"
)

# 发送到 litellm 代理上设置的模型请求，`litellm --model`
response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages = [
        {
            "role": "user",
            "content": "this is a test request, write a short poem"
        }
    ],
    extra_body={ # 传递任何特定于提供者的参数，如果 openai 不支持，https://docs.litellm.ai/docs/completion/input#provider-specific-params
        "metadata": { # 👈 用于记录额外参数（例如发送到 langfuse）
            "project_id": "my-special-project"
        }
    }
)

print(response)

更多示例，点击这里

完整 API 规范​

以下是您可以在 braintrust 请求的元数据中传递的所有内容

braintrust_* - 任何以 braintrust_ 开头的元数据字段都将作为元数据传递给日志记录请求

project_id - 设置 braintrust 调用的项目 ID。默认值为 litellm。