LLM 响应评分
有时根据生成文本的质量为LLM响应分配一个评分等级是很有用的。
在这个示例中，您正在评估一个大语言模型（LLM）总结文档的能力。输入数据包括文档和摘要，然后您需要以1到5的评分标准对摘要进行打分。
关于如何使用此模板与Label Studio SDK的教程，请参阅Evaluate LLM Responses。
配置标注界面
创建项目并设置以下标注配置：
<View>
<Style>
.root {
font-family: Arial, sans-serif;
margin: 0;
padding: 0;
display: flex;
flex-direction: column;
height: 100vh;
}
.container {
display: flex;
flex: 1;
}
.block {
flex: 1;
padding: 20px;
box-sizing: border-box;
}
.scrollable {
overflow-y: auto;
height: calc(100vh - 80px); /* Adjust height to accommodate header and footer */
}
.long-document {
background-color: #f9f9f9;
border-right: 1px solid #ddd;
}
.short-summary {
background-color: #f1f1f1;
}
.summary-rating {
padding: 20px;
background-color: #c3bcbc;
border-top: 1px solid #ddd;
text-align: center;
}
h2 {
margin-top: 0;
}
</Style>
<View className="root">
<View className="container">
<View className="block long-document scrollable">
<Header value="Long Document"/>
<Text name="document" value="$document"/>
</View>
<View className="block short-summary">
<Header value="Short Summary"/>
<Text name="summary" value="$summary"/>
</View>
</View>
<View className="summary-rating">
<Header value="Rate the Document Summary:"/>
<Rating name="rating" toName="summary" required="true"/>
</View>
</View>
</View>此配置包含以下元素：
<View>- All labeling configurations must include a baseViewtag. In this configuration, theViewtag is used to configure the display of blocks, similar to the div tag in HTML. It helps in organizing the layout of the labeling interface.<Style>-Style标签用于定义应用于View内元素的CSS样式。在此配置中，它为标注界面布局的各个部分设置了不同类的样式。Header标签用于在标注界面中显示标题或题头。它通过value属性设置标题文本。例如，Text标签用于显示输入数据提供的文本内容。根据下面的示例输入数据，文本块将显示JSON中document或summary键对应的信息。您可能需要调整该值以匹配您自己的JSON结构。
输入数据
使用上述配置时，您需要将输入数据结构化为包含document和summary元素。例如：
[
{
"data": {
"document": [
"Opossums, commonly known as possums in North America, are marsupials found primarily in the Americas. The most well-known species is the Virginia opossum (Didelphis virginiana), which ranges from Central America and the eastern United States to southern Canada. These adaptable creatures are known for their ability to thrive in a variety of environments, including both rural and urban areas. Opossums are also found in South America, where different species inhabit a range of ecosystems, from tropical rainforests to temperate forests.",
"Opossums are highly adaptable in terms of habitat, often residing in woodlands, farmland, and even suburban backyards. They typically seek shelter in hollow trees, abandoned burrows, or any dark, enclosed space they can find. Opossums are nocturnal and omnivorous, with a diet that includes fruits, insects, small animals, and even carrion. Their opportunistic feeding habits contribute to their resilience and ability to live in close proximity to human settlements.",
"In terms of behavior, opossums are solitary and nomadic, often moving to different locations in search of food. They are known for their unique defense mechanism of 'playing dead' or 'playing possum' when threatened, which involves mimicking the appearance and smell of a sick or dead animal to deter predators. Opossums have relatively short lifespans, typically living only 2 to 4 years in the wild. Despite their short lives, they reproduce quickly, with females giving birth to large litters of up to 20 young, although not all offspring typically survive to maturity.",
"In popular culture, opossums often appear as symbols of resilience and survival due to their hardy nature and ability to adapt to various environments. They are sometimes depicted in a comical or misunderstood light, given their nocturnal habits and somewhat disheveled appearance. Despite this, they play a crucial role in the ecosystem by controlling insect and rodent populations and cleaning up carrion. Opossums have been featured in various forms of media, from cartoons and children's books to movies, often emphasizing their unique behaviors and survival strategies."
],
"summary": "Opossums, primarily found in the Americas, are adaptable marsupials known for thriving in diverse environments, from rural to urban areas. They are nocturnal and omnivorous, often seeking shelter in dark, enclosed spaces and employing a unique defense mechanism of 'playing dead' to deter predators. In popular culture, opossums symbolize resilience and survival, playing a crucial role in ecosystems by controlling insect and rodent populations and cleaning up carrion."
}
}
]