Oracle云基础设施数据科学¶

Oracle云基础设施(OCI)数据科学是一个完全托管、无服务器的平台，供数据科学团队在Oracle云基础设施中构建、训练和管理机器学习模型。

它提供AI快速操作功能，可用于在OCI数据科学中部署、评估和微调基础大语言模型。AI快速操作面向希望快速利用AI能力的用户，旨在通过提供一个简化、无需编码且高效的基础模型操作环境，将基础模型的覆盖范围扩展到更广泛的用户群体。AI快速操作可从数据科学笔记本中访问。

关于如何在OCI数据科学中使用AI快速操作部署LLM模型的详细文档，请参阅此处和此处。

本笔记本介绍如何在LlamaIndex中使用OCI的数据科学模型。

设置¶

如果你在Colab上打开这个Notebook，你可能需要安装LlamaIndex 🦙。

In [ ]:

Copied!

%pip install llama-index-llms-oci-data-science
%pip install llama-index-llms-oci-data-science

In [ ]:

Copied!

!pip install llama-index
!pip install llama-index

您还需要安装 oracle-ads SDK。

In [ ]:

Copied!

!pip install -U oracle-ads
!pip install -U oracle-ads

身份验证¶

LlamaIndex支持的身份验证方法与其他OCI服务相同，遵循标准SDK身份验证方法，具体包括API密钥、会话令牌、实例主体和资源主体。更多详情可查阅此处。请确保具备访问OCI数据科学模型部署端点所需的策略。oracle-ads工具有助于简化OCI数据科学中的身份验证流程。

基本用法¶

使用OCI数据科学AI提供的LLM与LlamaIndex配合，只需使用您的数据科学模型部署端点和模型ID初始化OCIDataScience接口。默认情况下，AI快速操作中部署的所有模型都会获得odsc-model ID。不过这个ID可以在部署过程中更改。

使用提示调用`complete`方法¶

In [ ]:

Copied!





import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = llm.complete("Tell me a joke")

print(response)
import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https:///predict",
)
response = llm.complete("Tell me a joke")

print(response)

使用消息列表调用`chat`¶

In [ ]:

Copied!





import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = llm.chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

print(response)
import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https:///predict",
)
response = llm.chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

print(response)

流式传输¶

使用 `stream_complete` 端点¶

In [ ]:

Copied!





import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)

for chunk in llm.stream_complete("Tell me a joke"):
    print(chunk.delta, end="")
import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https:///predict",
)

for chunk in llm.stream_complete("Tell me a joke"):
    print(chunk.delta, end="")

使用 `stream_chat` 端点¶

In [ ]:

Copied!





import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = llm.stream_chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

for chunk in response:
    print(chunk.delta, end="")
import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https:///predict",
)
response = llm.stream_chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

for chunk in response:
    print(chunk.delta, end="")

异步¶

使用提示调用`acomplete`¶

In [ ]:

Copied!





import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = await llm.acomplete("Tell me a joke")

print(response)
import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https:///predict",
)
response = await llm.acomplete("Tell me a joke")

print(response)

使用消息列表调用`achat`¶

In [ ]:

Copied!





import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = await llm.achat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

print(response)
import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https:///predict",
)
response = await llm.achat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

print(response)

使用 `astream_complete` 端点¶

In [ ]:

Copied!





import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)

async for chunk in await llm.astream_complete("Tell me a joke"):
    print(chunk.delta, end="")
import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https:///predict",
)

async for chunk in await llm.astream_complete("Tell me a joke"):
    print(chunk.delta, end="")

使用 `astream_chat` 端点¶

In [ ]:

Copied!





import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
)
response = await llm.stream_chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

async for chunk in response:
    print(chunk.delta, end="")
import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.base.llms.types import ChatMessage

ads.set_auth(auth="security_token", profile="")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https:///predict",
)
response = await llm.stream_chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
        ChatMessage(
            role="assistant", content="Why did the chicken cross the road?"
        ),
        ChatMessage(role="user", content="I don't know, why?"),
    ]
)

async for chunk in response:
    print(chunk.delta, end="")

配置模型¶

In [ ]:

Copied!





import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
    temperature=0.2,
    max_tokens=500,
    timeout=120,
    context_window=2500,
    additional_kwargs={
        "top_p": 0.75,
        "logprobs": True,
        "top_logprobs": 3,
    },
)
response = llm.chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
    ]
)
print(response)
import ads
from llama_index.llms.oci_data_science import OCIDataScience

ads.set_auth(auth="security_token", profile="")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https:///predict",
    temperature=0.2,
    max_tokens=500,
    timeout=120,
    context_window=2500,
    additional_kwargs={
        "top_p": 0.75,
        "logprobs": True,
        "top_logprobs": 3,
    },
)
response = llm.chat(
    [
        ChatMessage(role="user", content="Tell me a joke"),
    ]
)
print(response)

函数调用¶

AI快速操作提供了预构建的服务容器，使得部署和服务大型语言模型变得非常简单。服务容器中使用了vLLM（一个针对LLM的高吞吐量和内存高效的推理与服务引擎）或TGI（一个为流行开源LLM设计的高性能文本生成服务器）之一来托管模型，创建的端点支持OpenAI API协议。这使得模型部署可以作为使用OpenAI API的应用程序的直接替代品。如果部署的模型支持函数调用，那么通过与LlamaIndex工具的集成，通过llm上的predict_and_call函数可以附加任何工具，并让LLM决定调用哪些工具（如果有的话）。

In [ ]:

Copied!





import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.tools import FunctionTool

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
    temperature=0.2,
    max_tokens=500,
    timeout=120,
    context_window=2500,
    additional_kwargs={
        "top_p": 0.75,
        "logprobs": True,
        "top_logprobs": 3,
    },
)


def multiply(a: float, b: float) -> float:
    print(f"---> {a} * {b}")
    return a * b


def add(a: float, b: float) -> float:
    print(f"---> {a} + {b}")
    return a + b


def subtract(a: float, b: float) -> float:
    print(f"---> {a} - {b}")
    return a - b


def divide(a: float, b: float) -> float:
    print(f"---> {a} / {b}")
    return a / b


multiply_tool = FunctionTool.from_defaults(fn=multiply)
add_tool = FunctionTool.from_defaults(fn=add)
sub_tool = FunctionTool.from_defaults(fn=subtract)
divide_tool = FunctionTool.from_defaults(fn=divide)

response = llm.predict_and_call(
    [multiply_tool, add_tool, sub_tool, divide_tool],
    user_msg="Calculate the result of `8 + 2 - 6`.",
    verbose=True,
)

print(response)
import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.tools import FunctionTool

ads.set_auth(auth="security_token", profile="")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https:///predict",
    temperature=0.2,
    max_tokens=500,
    timeout=120,
    context_window=2500,
    additional_kwargs={
        "top_p": 0.75,
        "logprobs": True,
        "top_logprobs": 3,
    },
)


def multiply(a: float, b: float) -> float:
    print(f"---> {a} * {b}")
    return a * b


def add(a: float, b: float) -> float:
    print(f"---> {a} + {b}")
    return a + b


def subtract(a: float, b: float) -> float:
    print(f"---> {a} - {b}")
    return a - b


def divide(a: float, b: float) -> float:
    print(f"---> {a} / {b}")
    return a / b


multiply_tool = FunctionTool.from_defaults(fn=multiply)
add_tool = FunctionTool.from_defaults(fn=add)
sub_tool = FunctionTool.from_defaults(fn=subtract)
divide_tool = FunctionTool.from_defaults(fn=divide)

response = llm.predict_and_call(
    [multiply_tool, add_tool, sub_tool, divide_tool],
    user_msg="Calculate the result of `8 + 2 - 6`.",
    verbose=True,
)

print(response)

使用 `FunctionCallingAgent`¶

In [ ]:

Copied!





import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.tools import FunctionTool
from llama_index.core.agent import FunctionCallingAgent

ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https://<MD_OCID>/predict",
    temperature=0.2,
    max_tokens=500,
    timeout=120,
    context_window=2500,
    additional_kwargs={
        "top_p": 0.75,
        "logprobs": True,
        "top_logprobs": 3,
    },
)


def multiply(a: float, b: float) -> float:
    print(f"---> {a} * {b}")
    return a * b


def add(a: float, b: float) -> float:
    print(f"---> {a} + {b}")
    return a + b


def subtract(a: float, b: float) -> float:
    print(f"---> {a} - {b}")
    return a - b


def divide(a: float, b: float) -> float:
    print(f"---> {a} / {b}")
    return a / b


multiply_tool = FunctionTool.from_defaults(fn=multiply)
add_tool = FunctionTool.from_defaults(fn=add)
sub_tool = FunctionTool.from_defaults(fn=subtract)
divide_tool = FunctionTool.from_defaults(fn=divide)

agent = FunctionCallingAgent.from_tools(
    tools=[multiply_tool, add_tool, sub_tool, divide_tool],
    llm=llm,
    verbose=True,
)
response = agent.chat(
    "Calculate the result of `8 + 2 - 6`. Use tools. Return the calculated result."
)

print(response)
import ads
from llama_index.llms.oci_data_science import OCIDataScience
from llama_index.core.tools import FunctionTool
from llama_index.core.agent import FunctionCallingAgent

ads.set_auth(auth="security_token", profile="")

llm = OCIDataScience(
    model="odsc-llm",
    endpoint="https:///predict",
    temperature=0.2,
    max_tokens=500,
    timeout=120,
    context_window=2500,
    additional_kwargs={
        "top_p": 0.75,
        "logprobs": True,
        "top_logprobs": 3,
    },
)


def multiply(a: float, b: float) -> float:
    print(f"---> {a} * {b}")
    return a * b


def add(a: float, b: float) -> float:
    print(f"---> {a} + {b}")
    return a + b


def subtract(a: float, b: float) -> float:
    print(f"---> {a} - {b}")
    return a - b


def divide(a: float, b: float) -> float:
    print(f"---> {a} / {b}")
    return a / b


multiply_tool = FunctionTool.from_defaults(fn=multiply)
add_tool = FunctionTool.from_defaults(fn=add)
sub_tool = FunctionTool.from_defaults(fn=subtract)
divide_tool = FunctionTool.from_defaults(fn=divide)

agent = FunctionCallingAgent.from_tools(
    tools=[multiply_tool, add_tool, sub_tool, divide_tool],
    llm=llm,
    verbose=True,
)
response = agent.chat(
    "Calculate the result of `8 + 2 - 6`. Use tools. Return the calculated result."
)

print(response)

Oracle云基础设施数据科学¶

设置¶

身份验证¶

基本用法¶

使用提示调用complete方法¶

使用消息列表调用chat¶

流式传输¶

使用 stream_complete 端点¶

使用 stream_chat 端点¶

异步¶

使用提示调用acomplete¶

使用消息列表调用achat¶

使用 astream_complete 端点¶

使用 astream_chat 端点¶

配置模型¶

函数调用¶

使用 FunctionCallingAgent¶

使用提示调用`complete`方法¶

使用消息列表调用`chat`¶

使用 `stream_complete` 端点¶

使用 `stream_chat` 端点¶

使用提示调用`acomplete`¶

使用消息列表调用`achat`¶

使用 `astream_complete` 端点¶

使用 `astream_chat` 端点¶

使用 `FunctionCallingAgent`¶