Skip to main content

预固定助手消息

由以下支持：

  • Deepseek
  • Mistral
  • Anthropic
{
  "role": "assistant", 
  "content": "..", 
  ...
  "prefix": true # 👈 关键变化
}

快速开始​

from litellm import completion
import os 

os.environ["DEEPSEEK_API_KEY"] = ""

response = completion(
  model="deepseek/deepseek-chat",
  messages=[
    {"role": "user", "content": "谁赢得了2022年的世界杯？"},
    {"role": "assistant", "content": "阿根廷", "prefix": True}
  ]
)
print(response.choices[0].message.content)
curl http://0.0.0.0:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_KEY" \
  -d '{
    "model": "deepseek/deepseek-chat",
    "messages": [
      {
        "role": "user",
        "content": "谁赢得了2022年的世界杯？"
      },
      {
        "role": "assistant", 
        "content": "阿根廷", "prefix": true
      }
    ]
}'

预期响应

{
    "id": "3b66124d79a708e10c603496b363574c",
    "choices": [
        {
            "finish_reason": "stop",
            "index": 0,
            "message": {
                "content": "赢得了2022年FIFA世界杯。",
                "role": "assistant",
                "tool_calls": null,
                "function_call": null
            }
        }
    ],
    "created": 1723323084,
    "model": "deepseek/deepseek-chat",
    "object": "chat.completion",
    "system_fingerprint": "fp_7e0991cad4",
    "usage": {
        "completion_tokens": 12,
        "prompt_tokens": 16,
        "total_tokens": 28,
    },
    "service_tier": null
}

检查模型支持​

调用 litellm.get_model_info 来检查一个模型/提供商是否支持 response_format。

from litellm import get_model_info

params = get_model_info(model="deepseek/deepseek-chat")

assert params["supports_assistant_prefill"] is True

调用 /model/info 端点以获取模型列表及其支持的参数。

curl -X GET 'http://0.0.0.0:4000/v1/model/info' \
-H 'Authorization: Bearer $LITELLM_KEY' \
优云智算