Skip to main content

火山引擎 (Volcengine)

https://www.volcengine.com/docs/82379/1263482

tip

我们支持所有火山引擎NIM模型，只需在发送litellm请求时设置model=volcengine/<任何火山引擎上的模型>作为前缀

API密钥​

# 环境变量
os.environ['VOLCENGINE_API_KEY']

示例用法​

from litellm import completion
import os

os.environ['VOLCENGINE_API_KEY'] = ""
response = completion(
    model="volcengine/<我们的端点ID>",
    messages=[
        {
            "role": "user",
            "content": "今天波士顿的天气如何？以华氏度为单位。",
        }
    ],
    temperature=0.2,        # 可选
    top_p=0.9,              # 可选
    frequency_penalty=0.1,  # 可选
    presence_penalty=0.1,   # 可选
    max_tokens=10,          # 可选
    stop=["\n\n"],          # 可选
)
print(response)

示例用法 - 流式传输​

from litellm import completion
import os

os.environ['VOLCENGINE_API_KEY'] = ""
response = completion(
    model="volcengine/<我们的端点ID>",
    messages=[
        {
            "role": "user",
            "content": "今天波士顿的天气如何？以华氏度为单位。",
        }
    ],
    stream=True,
    temperature=0.2,        # 可选
    top_p=0.9,              # 可选
    frequency_penalty=0.1,  # 可选
    presence_penalty=0.1,   # 可选
    max_tokens=10,          # 可选
    stop=["\n\n"],          # 可选
)

for chunk in response:
    print(chunk)

支持的模型 - 💥 所有火山引擎NIM模型均受支持！​

我们支持所有volcengine模型，只需在发送完成请求时设置volcengine/<我们的端点ID>作为前缀

示例用法 - LiteLLM代理​

config.yaml设置​

model_list:
  - model_name: volcengine-model
    litellm_params:
      model: volcengine/<我们的端点ID>
      api_key: os.environ/VOLCENGINE_API_KEY

发送请求​

curl --location 'http://localhost:4000/chat/completions' \
    --header 'Authorization: Bearer sk-1234' \
    --header 'Content-Type: application/json' \
    --data '{
    "model": "volcengine-model",
    "messages": [
        {
        "role": "user",
        "content": "这是我的API密钥。openai_api_key=sk-1234"
        }
    ]
}'