Azure Blob存储中的图像

可以使用Azure Node.JS SDK从Azure Blog Storage下载图像并在提示中使用。defImages函数支持node.js [Buffer]类型。

配置

安装 @azure/storage-blob 和 @azure/identity 包。

npm install -D @azure/storage-blob @azure/identity

请确保使用Azure CLI登录并设置订阅。

az login

读取二进制大对象(blobs)

打开与Azure Blob Storage的连接并获取容器客户端。我们从env.vars对象中解构出account和container，以便可以通过cli进行设置。

import { BlobServiceClient } from "@azure/storage-blob"
import { DefaultAzureCredential } from "@azure/identity"

const { account = "myblobs", container = "myimages" } = env.vars
const blobServiceClient = new BlobServiceClient(
    `https://${account}.blob.core.windows.net`,
    new DefaultAzureCredential()
)
const containerClient = blobServiceClient.getContainerClient(container)

如果没有特定的blob对象，您可以遍历这些blob，并将它们下载到缓冲区(buf)中。

import { buffer } from "node:stream/consumers"

for await (const blob of containerClient.listBlobsFlat()) {
    const blockBlobClient = containerClient.getBlockBlobClient(blob.name)
    const downloadBlockBlobResponse = await blockBlobClient.download(0)
    const body = await downloadBlockBlobResponse.readableStreamBody
    const image = await buffer(body)
    ...

在提示中使用图像

image缓冲区可以通过defImages传递，以便在提示中使用。

    defImages(image, { detail: "low" })

但由于图像可能"体积较大"，您很可能需要使用 inline prompts将其拆分为更小的查询。(注意_.的使用)

for await (const blob of containerClient.listBlobsFlat()) {
    ...
    const res = await runPrompt(_ => {
        _.defImages(image, { detail: "low" })
        _.$`Describe the image.`
    })
    // res contains the LLM response for the inner prompt
    ...

结果摘要

为了总结所有图像，我们使用def函数存储每个图像的摘要，并添加提示来总结描述。

    ...
    def("IMAGES_SUMMARY", { filename: blob.name, content: res.text })
}
$`Summarize IMAGES_SUMMARY.`

完整源代码

import { BlobServiceClient } from "@azure/storage-blob"
import { DefaultAzureCredential } from "@azure/identity"
import { buffer } from "node:stream/consumers"

script({
    parameters: {
        account: {
            description: "Azure Storage Account Name",
            default: "genaiscript",
            type: "string",
        },
        container: {
            description: "Azure Storage Container Name",
            default: "images",
            type: "string",
        },
    },
})

const { account, container } = env.vars
const url = `https://${account}.blob.core.windows.net`
console.log(`analyzing images in ${account}/${container} at ${url}`)
const blobServiceClient = new BlobServiceClient(
    url,
    new DefaultAzureCredential()
)
const containerClient = blobServiceClient.getContainerClient(container)
for await (const blob of containerClient.listBlobsFlat()) {
    console.log(`blob: ` + blob.name)
    const blockBlobClient = containerClient.getBlockBlobClient(blob.name)
    const downloadBlockBlobResponse = await blockBlobClient.download(0)
    const body = await downloadBlockBlobResponse.readableStreamBody
    const image = await buffer(body)

    const res = await runPrompt(_ => {
        _.defImages(image, { detail: "low" })
        _.$`Describe the images.`
    })

    def("IMAGES_SUMMARY", { filename: blob.name, content: res.text })
}

$`Summarize IMAGES_SUMMARY.`