跳到主要内容

LM Studio

Open In Colab Open on GitHub

本笔记本展示了如何使用AutoGen与多个本地模型结合使用LM Studio的多模型服务功能，该功能自LM Studio版本0.2.17起可用。

要在LM Studio中使用多模型服务功能，您可以在“Playground”标签中启动“多模型会话”。然后选择相关模型进行加载。模型加载完成后，您可以点击“启动服务器”以启动多模型服务。这些模型将在一个本地托管的OpenAI兼容端点上可用。

两个代理聊天​

在这个示例中，我们使用两个不同的本地模型，Phi-2 和 Gemma it，创建了两个代理之间的喜剧对话。

我们首先为模型创建配置。

gemma = {
    "config_list": [
        {
            "model": "lmstudio-ai/gemma-2b-it-GGUF/gemma-2b-it-q8_0.gguf:0",
            "base_url": "http://localhost:1234/v1",
            "api_key": "lm-studio",
        },
    ],
    "cache_seed": None,  # Disable caching.
}

phi2 = {
    "config_list": [
        {
            "model": "TheBloke/phi-2-GGUF/phi-2.Q4_K_S.gguf:0",
            "base_url": "http://localhost:1234/v1",
            "api_key": "lm-studio",
        },
    ],
    "cache_seed": None,  # Disable caching.
}

现在我们创建两个代理，每个模型一个。

from autogen import ConversableAgent

jack = ConversableAgent(
    "Jack (Phi-2)",
    llm_config=phi2,
    system_message="Your name is Jack and you are a comedian in a two-person comedy show.",
)
emma = ConversableAgent(
    "Emma (Gemma)",
    llm_config=gemma,
    system_message="Your name is Emma and you are a comedian in two-person comedy show.",
)

现在我们开始对话。

chat_result = jack.initiate_chat(emma, message="Emma, tell me a joke.", max_turns=2)
Jack (Phi-2) (to Emma (Gemma)):

Emma, tell me a joke.

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Emma (Gemma) (to Jack (Phi-2)):

Sure! Here's a joke for you:

What do you call a comedian who's too emotional?

An emotional wreck!

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Jack (Phi-2) (to Emma (Gemma)):

LOL, that's a good one, Jack! You're hilarious. 😂👏👏


--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
Emma (Gemma) (to Jack (Phi-2)):

Thank you! I'm just trying to make people laugh, you know? And to help them forget about the troubles of the world for a while.

--------------------------------------------------------------------------------