🆕 Databricks
LiteLLM 支持 Databricks 上的所有模型
tip
我们支持所有 Databricks 模型，只需在发送 litellm 请求时将 model=databricks/<任意-databricks-模型> 作为前缀
使用方法
- SDK
- 代理
环境变量
import os
os.environ["DATABRICKS_API_KEY"] = ""
os.environ["DATABRICKS_API_BASE"] = ""
示例调用
from litellm import completion
import os
## 设置环境变量
os.environ["DATABRICKS_API_KEY"] = "databricks key"
os.environ["DATABRICKS_API_BASE"] = "databricks base url" # 例如：https://adb-3064715882934586.6.azuredatabricks.net/serving-endpoints
# Databricks dbrx-instruct 调用
response = completion(
model="databricks/databricks-dbrx-instruct",
messages = [{ "content": "Hello, how are you?","role": "user"}]
)
将模型添加到 config.yaml
model_list:
- model_name: dbrx-instruct
litellm_params:
model: databricks/databricks-dbrx-instruct
api_key: os.environ/DATABRICKS_API_KEY
api_base: os.environ/DATABRICKS_API_BASE
启动代理
$ litellm --config /path/to/config.yaml --debug向 LiteLLM 代理服务器发送请求
- OpenAI Python v1.0.0+
- curl
import openai
client = openai.OpenAI(
api_key="sk-1234", # 如果使用虚拟密钥，请传递 litellm 代理密钥
base_url="http://0.0.0.0:4000" # litellm-proxy-base url
)
response = client.chat.completions.create(
model="dbrx-instruct",
messages = [
{
"role": "system",
"content": "Be a good human!"
},
{
"role": "user",
"content": "What do you know about earth?"
}
]
)
print(response)curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Authorization: Bearer sk-1234' \
--header 'Content-Type: application/json' \
--data '{
"model": "dbrx-instruct",
"messages": [
{
"role": "system",
"content": "Be a good human!"
},
{
"role": "user",
"content": "What do you know about earth?"
}
],
}'
传递额外参数 - max_tokens, temperature
查看所有支持的 litellm.completion 参数 这里
# !pip install litellm
from litellm import completion
import os
## 设置环境变量
os.environ["DATABRICKS_API_KEY"] = "databricks key"
os.environ["DATABRICKS_API_BASE"] = "databricks api base"
# databricks dbrx 调用
response = completion(
model="databricks/databricks-dbrx-instruct",
messages = [{ "content": "Hello, how are you?","role": "user"}],
max_tokens=20,
temperature=0.5
)
代理
model_list:
- model_name: llama-3
litellm_params:
model: databricks/databricks-meta-llama-3-70b-instruct
api_key: os.environ/DATABRICKS_API_KEY
max_tokens: 20
temperature: 0.5
传递 Databricks 特定参数 - 'instruction'
对于嵌入模型，databricks 允许您传递额外的参数 'instruction'。完整规范
# !pip install litellm
from litellm import embedding
import os
## 设置环境变量
os.environ["DATABRICKS_API_KEY"] = "databricks key"
os.environ["DATABRICKS_API_BASE"] = "databricks url"
# Databricks bge-large-en 调用
response = litellm.embedding(
model="databricks/databricks-bge-large-en",
input=["good morning from litellm"],
instruction="Represent this sentence for searching relevant passages:",
)
代理
model_list:
- model_name: bge-large
litellm_params:
model: databricks/databricks-bge-large-en
api_key: os.environ/DATABRICKS_API_KEY
api_base: os.environ/DATABRICKS_API_BASE
instruction: "Represent this sentence for searching relevant passages:"
支持的 Databricks 聊天完成模型
tip
我们支持所有 Databricks 模型，只需在发送 litellm 请求时将 model=databricks/<任意-databricks-模型> 作为前缀
| 模型名称 | 命令 |
|---|---|
| databricks-meta-llama-3-1-70b-instruct | completion(model='databricks/databricks-meta-llama-3-1-70b-instruct', messages=messages) |
| databricks-meta-llama-3-1-405b-instruct | completion(model='databricks/databricks-meta-llama-3-1-405b-instruct', messages=messages) |
| databricks-dbrx-instruct | completion(model='databricks/databricks-dbrx-instruct', messages=messages) |
| databricks-meta-llama-3-70b-instruct | completion(model='databricks/databricks-meta-llama-3-70b-instruct', messages=messages) |
| databricks-llama-2-70b-chat | completion(model='databricks/databricks-llama-2-70b-chat', messages=messages) |
| databricks-mixtral-8x7b-instruct | completion(model='databricks/databricks-mixtral-8x7b-instruct', messages=messages) |
| databricks-mpt-30b-instruct | completion(model='databricks/databricks-mpt-30b-instruct', messages=messages) |
| databricks-mpt-7b-instruct | completion(model='databricks/databricks-mpt-7b-instruct', messages=messages) |
支持的Databricks嵌入模型
tip
我们支持所有Databricks模型，只需在发送litellm请求时将model=databricks/<任何在Databricks上的模型>设置为前缀
| 模型名称 | 命令 |
|---|---|
| databricks-bge-large-en | embedding(model='databricks/databricks-bge-large-en', messages=messages) |
| databricks-gte-large-en | embedding(model='databricks/databricks-gte-large-en', messages=messages) |
