从OpenAI迁移
Predibase支持OpenAI Chat Completions v1兼容的端点，这使得从OpenAI迁移到Predibase变得尽可能简单。
操作步骤
- 获取您的Predibase API令牌
- 对于
base_url，请获取您的租户ID（可在设置 > 我的个人资料页面找到）和部署名称（可在部署页面找到） - 如果在Predibase上使用经过微调的适配器，您将需要适配器ID（"adapter repo name"/"version number"），在提示时该ID会作为
model参数包含其中。
Python SDK 示例
如果您使用apply_chat_template微调了一个适配器，请查看"Chat Completions"选项卡。
- 基础模型
- 微调适配器
- 聊天补全
- 函数调用
from openai import OpenAI
api_token = "<PREDIBASE API TOKEN>"
tenant_id = "<PREDIBASE TENANT ID>"
model_name = "<DEPLOYMENT NAME>" # Ex. "mistral-7b"
base_url = f"https://serving.app.predibase.com/{tenant_id}/deployments/v2/llms/{model_name}/v1"
client = OpenAI(
api_key=api_token,
base_url=base_url,
)
completion = client.completions.create(
model="",
prompt="How many helicopters can a human eat in one sitting?",
max_tokens=100,
)
print("Completion result:", completion.choices[0].text)
from openai import OpenAI
api_token = "<PREDIBASE API TOKEN>"
tenant_id = "<PREDIBASE TENANT ID>"
model_name = "<DEPLOYMENT NAME>" # Ex. "mistral-7b"
adapter = "<ADAPTER REPO NAME>/<VERSION NUMBER>" # Ex. "adapter-repo/1"
base_url = f"https://serving.app.predibase.com/{tenant_id}/deployments/v2/llms/{model_name}/v1"
client = OpenAI(
api_key=api_token,
base_url=base_url,
)
completion = client.completions.create(
model=adapter,
prompt="How many helicopters can a human eat in one sitting?",
max_tokens=100,
)
print("Completion result:", completion.choices[0].text)
OpenAI Chat Completions 会自动将聊天模板应用到您的请求中。
from openai import OpenAI
api_token = "<PREDIBASE API TOKEN>"
tenant_id = "<PREDIBASE TENANT ID>"
model_name = "<DEPLOYMENT NAME>" # Ex. "mistral-7b"
adapter = "<ADAPTER REPO NAME>/<VERSION NUMBER>" # Ex. "adapter-repo/1"
base_url = f"https://serving.app.predibase.com/{tenant_id}/deployments/v2/llms/{model_name}/v1"
client = OpenAI(
api_key=api_token,
base_url=base_url,
)
content = "How many helicopters can a human eat in one sitting?"
completion = client.chat.completions.create(
model=adapter,
messages=[{"role": "user", "content": content}],
max_tokens=100,
)
print("Completion result:", completion.choices[0].text)
OpenAI Chat Completions 也可以用来调用预定义的工具
from openai import OpenAI
api_token = "<PREDIBASE API TOKEN>"
tenant_id = "<PREDIBASE TENANT ID>"
model_name = "<DEPLOYMENT NAME>" # Ex. "mistral-7b"
adapter = "<ADAPTER REPO NAME>/<VERSION NUMBER>" # Ex. "adapter-repo/1"
base_url = f"https://serving.app.predibase.com/{tenant_id}/deployments/v2/llms/{model_name}/v1"
client = OpenAI(
api_key=api_token,
base_url=base_url,
)
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"}
},
},
},
}
]
content = "What's the weather like right now in California?"
completion = client.chat.completions.create(
model=adapter,
messages=[{"role": "user", "content": content}],
max_tokens=100,
tools=tools,
)
print("Completion result:", completion.choices[0].text)
流式处理
- 基础模型
- 微调适配器
from openai import OpenAI
api_token = "<PREDIBASE API TOKEN>"
tenant_id = "<PREDIBASE TENANT ID>"
model_name = "<DEPLOYMENT NAME>" # Ex. "mistral-7b"
base_url = f"https://serving.app.predibase.com/{tenant_id}/deployments/v2/llms/{model_name}/v1"
client = OpenAI(
api_key=api_token,
base_url=base_url,
)
# Prompt the base model
completion_stream = client.completions.create(
model="",
prompt="How many helicopters can a human eat in one sitting?",
max_tokens=100,
stream=True
)
text = []
for message in completion_stream:
print(message)
text.append(message.choices[0].text)
print("".join(text))
from openai import OpenAI
api_token = "<PREDIBASE API TOKEN>"
tenant_id = "<PREDIBASE TENANT ID>"
model_name = "<DEPLOYMENT NAME>" # Ex. "mistral-7b"
adapter = "<ADAPTER REPO NAME>/<VERSION NUMBER>" # Ex. "adapter-repo/1"
base_url = f"https://serving.app.predibase.com/{tenant_id}/deployments/v2/llms/{model_name}/v1"
client = OpenAI(
api_key=api_token,
base_url=base_url,
)
# Prompt the base model
completion_stream = client.completions.create(
model=adapter,
prompt="How many helicopters can a human eat in one sitting?",
max_tokens=100,
stream=True
)
response = []
for message in completion_stream:
token = message.choices[0].text
response.append(token)
print(token, end='')
print("".join(response))
REST API 示例
export PREDIBASE_API_TOKEN="<YOUR TOKEN HERE>"
export PREDIBASE_ENDPOINT="<YOUR ENDPOINT HERE>"
#PREDIBASE_ENDPOINT_EXAMPLE: https://serving.app.predibase.com/{tenantID}/deployments/v2/llms/mistral-7b-instruct
curl -i $PREDIBASE_ENDPOINT/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${PREDIBASE_API_TOKEN}" \
-d '{"model": "", "messages": [{ "role": "user", "content": "How many helicopters can a human eat in one sitting?"}],"max_tokens": 100}'