gpustack 聊天
与大型语言模型对话。
gpustack chat model [prompt]
位置参数
| 名称 | 描述 |
|---|---|
| model | 用于聊天的模型。 |
| prompt | 发送给模型的提示语。[可选] |
单次提示对话
如果提供了提示，它会执行一次性推理。例如：
gpustack chat llama3 "tell me a joke."
示例输出：
Why couldn't the bicycle stand up by itself?
Because it was two-tired!
交互式聊天
如果未提供prompt参数，您可以与大型语言模型进行交互式聊天。例如：
gpustack chat llama3
示例输出：
>tell me a joke.
Here's one:
Why couldn't the bicycle stand up by itself?
(wait for it...)
Because it was two-tired!
Hope that made you smile!
>Do you have a better one?
Here's another one:
Why did the scarecrow win an award?
(think about it for a sec...)
Because he was outstanding in his field!
Hope that one stuck with you!
Do you want to hear another one?
>\quit
交互式命令
以下是交互式聊天中可用的命令：
Commands:
\q or \quit - Quit the chat
\c or \clear - Clear chat context in prompt
\? or \h or \help - Print this help message
连接到外部GPUStack服务器
如果您不是在服务器节点上运行gpustack chat，或者您正在自定义主机或端口上提供服务，则应提供以下环境变量：
| 名称 | 描述 |
|---|---|
| GPUSTACK_SERVER_URL | URL of the GPUStack server, e.g., http://your_host_ip. |
| GPUSTACK_API_KEY | GPUStack API密钥。 |