跳转到内容

Nebius 大语言模型

本笔记本演示了如何将 Nebius AI Studio 的 LLMs 与 LlamaIndex 结合使用。Nebius AI Studio 实现了所有可供商业使用的最先进 LLMs。

首先，让我们安装LlamaIndex和Nebius AI Studio的依赖项。

%pip install llama-index-llms-nebius llama-index

在下方从系统变量上传您的 Nebius AI Studio 密钥或直接插入。您可以通过免费注册 Nebius AI Studio 并在 API 密钥部分生成密钥来获取。

import os

NEBIUS_API_KEY = os.getenv("NEBIUS_API_KEY")  # NEBIUS_API_KEY = ""

from llama_index.llms.nebius import NebiusLLM

llm = NebiusLLM(
    api_key=NEBIUS_API_KEY, model="meta-llama/Llama-3.3-70B-Instruct-fast"
)

None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.

Call `complete` with a prompt

response = llm.complete("Amsterdam is the capital of ")
print(response)

The Netherlands! Amsterdam is indeed the capital and largest city of the Netherlands.

Call `chat` with a list of messages

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="system", content="You are a helpful AI assistant."),
    ChatMessage(
        role="user",
        content="Answer briefly: who is Wall-e?",
    ),
]
response = llm.chat(messages)
print(response)

assistant: WALL-E is a small waste-collecting robot and the main character in the 2008 Pixar animated film of the same name.

流式处理

Using `stream_complete` endpoint

response = llm.stream_complete("Amsterdam is the capital of ")
for r in response:
    print(r.delta, end="")

The Netherlands! Amsterdam is indeed the capital and largest city of the Netherlands.

使用 `stream_chat` 处理消息列表

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="system", content="You are a helpful AI assistant."),
    ChatMessage(
        role="user",
        content="Answer briefly: who is Wall-e?",
    ),
]
response = llm.stream_chat(messages)
for r in response:
    print(r.delta, end="")

WALL-E is a small waste-collecting robot and the main character in the 2008 Pixar animated film of the same name.