微调通过训练比提示中能容纳的更多示例来改进模型，使您能在广泛的任务上取得更好的结果。本笔记本提供了我们新的GPT-4o mini微调的分步指南。我们将使用RecipeNLG数据集进行实体提取，该数据集提供了各种食谱以及为每个食谱提取的通用配料列表。这是命名实体识别(NER)任务的常用数据集。
注意：GPT-4o mini微调功能现面向Tier 4和5使用层级的开发者开放。 您可以通过访问微调仪表板，点击"创建"并从基础模型下拉菜单中选择"gpt-4o-mini-2024-07-18"来开始微调GPT-4o mini。
我们将按照以下步骤进行：
Files端点。通过本教程，您将能够训练、评估并部署一个经过微调的 gpt-4o-mini-2024-07-18 模型。
# make sure to use the latest version of the openai python package
!pip install --upgrade --quiet openaiimport json
import openai
import os
import pandas as pd
from pprint import pprint
client = openai.OpenAI(
api_key=os.environ.get("OPENAI_API_KEY"),
organization="<org id>",
project="<project id>",
)微调在专注于特定领域时效果最佳。重要的是确保您的数据集既足够集中以便模型学习，又足够通用以避免遗漏未见过的示例。考虑到这一点，我们从RecipesNLG数据集中提取了一个子集，仅包含来自cookbooks.com的文档。
# Read in the dataset we'll use for this task.
# This will be the RecipesNLG dataset, which we've cleaned to only contain documents from www.cookbooks.com
recipe_df = pd.read_csv("data/cookbook_recipes_nlg_10k.csv")
recipe_df.head()| 标题 | 食材 | 步骤 | 链接 | 来源 | 命名实体识别 | |
|---|---|---|---|---|---|---|
| 0 | 免烤坚果饼干 | ["1杯紧压红糖", "1/2杯 eva... | ["在一个2夸脱的厚底锅中，混合红糖... | www.cookbooks.com/Recipe-Details.aspx?id=44874 | www.cookbooks.com | ["红糖", "牛奶", "香草", "坚果", "黄油... |
| 1 | Jewell Ball'S 鸡肉 | ["1小罐切碎的牛肉，切块", "4块去骨... | ["将切碎的牛肉铺在烤盘底部... | www.cookbooks.com/Recipe-Details.aspx?id=699419 | www.cookbooks.com | ["牛肉", "鸡胸肉", "奶油蘑菇汤... |
| 2 | 奶油玉米 | ["2 (16盎司)包冷冻玉米", "1 (8盎司)包... | ["将所有配料放入慢炖锅中混合。烹... | www.cookbooks.com/Recipe-Details.aspx?id=10570 | www.cookbooks.com | ["冷冻玉米", "奶油奶酪", "黄油", "大蒜... |
| 3 | 趣味鸡肉 | ["1只整鸡", "2罐(10.5盎司)... | ["将鸡肉煮熟去骨", "将鸡肉切成一口大小... | www.cookbooks.com/Recipe-Details.aspx?id=897570 | www.cookbooks.com | ["鸡肉", "鸡肉酱汁", "蘑菇奶油... |
| 4 | 瑞茜杯(糖果) | ["1杯花生酱", "3/4杯全麦饼干屑... | ["将前四种材料混合后压入... | www.cookbooks.com/Recipe-Details.aspx?id=659239 | www.cookbooks.com | ["花生酱", "全麦饼干屑", "黄油... |
我们将从准备数据开始。当使用ChatCompletion格式进行微调时，每个训练样本都是一个简单的messages列表。例如，一个条目可能如下所示：
[{'role': 'system',
'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'},
{'role': 'user',
'content': 'Title: No-Bake Nut Cookies\n\nIngredients: ["1 c. firmly packed brown sugar", "1/2 c. evaporated milk", "1/2 tsp. vanilla", "1/2 c. broken nuts (pecans)", "2 Tbsp. butter or margarine", "3 1/2 c. bite size shredded rice biscuits"]\n\nGeneric ingredients: '},
{'role': 'assistant',
'content': '["brown sugar", "milk", "vanilla", "nuts", "butter", "bite size shredded rice biscuits"]'}]在训练过程中，这段对话会被拆分，最后一条消息将作为模型生成的completion，其余messages则作为提示。构建训练示例时请考虑这一点——如果您的模型需要处理多轮对话，请提供具有代表性的示例，这样当对话内容扩展时模型才不会表现不佳。
请注意，目前每个训练示例有4096个token的限制。超过此长度的内容将在第4096个token处被截断。
system_message = "You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided."
def create_user_message(row):
return f"Title: {row['title']}\n\nIngredients: {row['ingredients']}\n\nGeneric ingredients: "
def prepare_example_conversation(row):
return {
"messages": [
{"role": "system", "content": system_message},
{"role": "user", "content": create_user_message(row)},
{"role": "assistant", "content": row["NER"]},
]
}
pprint(prepare_example_conversation(recipe_df.iloc[0])){'messages': [{'content': 'You are a helpful recipe assistant. You are to '
'extract the generic ingredients from each of the '
'recipes provided.',
'role': 'system'},
{'content': 'Title: No-Bake Nut Cookies\n'
'\n'
'Ingredients: ["1 c. firmly packed brown sugar", '
'"1/2 c. evaporated milk", "1/2 tsp. vanilla", "1/2 '
'c. broken nuts (pecans)", "2 Tbsp. butter or '
'margarine", "3 1/2 c. bite size shredded rice '
'biscuits"]\n'
'\n'
'Generic ingredients: ',
'role': 'user'},
{'content': '["brown sugar", "milk", "vanilla", "nuts", '
'"butter", "bite size shredded rice biscuits"]',
'role': 'assistant'}]}
现在让我们对数据集的一个子集执行此操作，作为我们的训练数据。你甚至可以从30-50个精心修剪的示例开始。随着训练集规模的增加，你应该会看到性能持续线性提升，但任务耗时也会相应增加。
# use the first 100 rows of the dataset for training
training_df = recipe_df.loc[0:100]
# apply the prepare_example_conversation function to each row of the training_df
training_data = training_df.apply(prepare_example_conversation, axis=1).tolist()
for example in training_data[:5]:
print(example){'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: No-Bake Nut Cookies\n\nIngredients: ["1 c. firmly packed brown sugar", "1/2 c. evaporated milk", "1/2 tsp. vanilla", "1/2 c. broken nuts (pecans)", "2 Tbsp. butter or margarine", "3 1/2 c. bite size shredded rice biscuits"]\n\nGeneric ingredients: '}, {'role': 'assistant', 'content': '["brown sugar", "milk", "vanilla", "nuts", "butter", "bite size shredded rice biscuits"]'}]}
{'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: Jewell Ball\'S Chicken\n\nIngredients: ["1 small jar chipped beef, cut up", "4 boned chicken breasts", "1 can cream of mushroom soup", "1 carton sour cream"]\n\nGeneric ingredients: '}, {'role': 'assistant', 'content': '["beef", "chicken breasts", "cream of mushroom soup", "sour cream"]'}]}
{'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: Creamy Corn\n\nIngredients: ["2 (16 oz.) pkg. frozen corn", "1 (8 oz.) pkg. cream cheese, cubed", "1/3 c. butter, cubed", "1/2 tsp. garlic powder", "1/2 tsp. salt", "1/4 tsp. pepper"]\n\nGeneric ingredients: '}, {'role': 'assistant', 'content': '["frozen corn", "cream cheese", "butter", "garlic powder", "salt", "pepper"]'}]}
{'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: Chicken Funny\n\nIngredients: ["1 large whole chicken", "2 (10 1/2 oz.) cans chicken gravy", "1 (10 1/2 oz.) can cream of mushroom soup", "1 (6 oz.) box Stove Top stuffing", "4 oz. shredded cheese"]\n\nGeneric ingredients: '}, {'role': 'assistant', 'content': '["chicken", "chicken gravy", "cream of mushroom soup", "shredded cheese"]'}]}
{'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: Reeses Cups(Candy) \n\nIngredients: ["1 c. peanut butter", "3/4 c. graham cracker crumbs", "1 c. melted butter", "1 lb. (3 1/2 c.) powdered sugar", "1 large pkg. chocolate chips"]\n\nGeneric ingredients: '}, {'role': 'assistant', 'content': '["peanut butter", "graham cracker crumbs", "butter", "powdered sugar", "chocolate chips"]'}]}
除了训练数据外，我们还可以选择性地提供验证数据，这将用于确保模型不会对您的训练集产生过拟合。
validation_df = recipe_df.loc[101:200]
validation_data = validation_df.apply(
prepare_example_conversation, axis=1).tolist()然后我们需要将数据保存为.jsonl文件，每行代表一个训练示例对话。
def write_jsonl(data_list: list, filename: str) -> None:
with open(filename, "w") as out:
for ddict in data_list:
jout = json.dumps(ddict) + "\n"
out.write(jout)training_file_name = "tmp_recipe_finetune_training.jsonl"
write_jsonl(training_data, training_file_name)
validation_file_name = "tmp_recipe_finetune_validation.jsonl"
write_jsonl(validation_data, validation_file_name)这是我们训练.jsonl文件的前5行内容：
# print the first 5 lines of the training file
!head -n 5 tmp_recipe_finetune_training.jsonl{"messages": [{"role": "system", "content": "You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided."}, {"role": "user", "content": "Title: No-Bake Nut Cookies\n\nIngredients: [\"1 c. firmly packed brown sugar\", \"1/2 c. evaporated milk\", \"1/2 tsp. vanilla\", \"1/2 c. broken nuts (pecans)\", \"2 Tbsp. butter or margarine\", \"3 1/2 c. bite size shredded rice biscuits\"]\n\nGeneric ingredients: "}, {"role": "assistant", "content": "[\"brown sugar\", \"milk\", \"vanilla\", \"nuts\", \"butter\", \"bite size shredded rice biscuits\"]"}]}
{"messages": [{"role": "system", "content": "You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided."}, {"role": "user", "content": "Title: Jewell Ball'S Chicken\n\nIngredients: [\"1 small jar chipped beef, cut up\", \"4 boned chicken breasts\", \"1 can cream of mushroom soup\", \"1 carton sour cream\"]\n\nGeneric ingredients: "}, {"role": "assistant", "content": "[\"beef\", \"chicken breasts\", \"cream of mushroom soup\", \"sour cream\"]"}]}
{"messages": [{"role": "system", "content": "You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided."}, {"role": "user", "content": "Title: Creamy Corn\n\nIngredients: [\"2 (16 oz.) pkg. frozen corn\", \"1 (8 oz.) pkg. cream cheese, cubed\", \"1/3 c. butter, cubed\", \"1/2 tsp. garlic powder\", \"1/2 tsp. salt\", \"1/4 tsp. pepper\"]\n\nGeneric ingredients: "}, {"role": "assistant", "content": "[\"frozen corn\", \"cream cheese\", \"butter\", \"garlic powder\", \"salt\", \"pepper\"]"}]}
{"messages": [{"role": "system", "content": "You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided."}, {"role": "user", "content": "Title: Chicken Funny\n\nIngredients: [\"1 large whole chicken\", \"2 (10 1/2 oz.) cans chicken gravy\", \"1 (10 1/2 oz.) can cream of mushroom soup\", \"1 (6 oz.) box Stove Top stuffing\", \"4 oz. shredded cheese\"]\n\nGeneric ingredients: "}, {"role": "assistant", "content": "[\"chicken\", \"chicken gravy\", \"cream of mushroom soup\", \"shredded cheese\"]"}]}
{"messages": [{"role": "system", "content": "You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided."}, {"role": "user", "content": "Title: Reeses Cups(Candy) \n\nIngredients: [\"1 c. peanut butter\", \"3/4 c. graham cracker crumbs\", \"1 c. melted butter\", \"1 lb. (3 1/2 c.) powdered sugar\", \"1 large pkg. chocolate chips\"]\n\nGeneric ingredients: "}, {"role": "assistant", "content": "[\"peanut butter\", \"graham cracker crumbs\", \"butter\", \"powdered sugar\", \"chocolate chips\"]"}]}
您现在可以将文件上传到我们的Files端点，供微调模型使用。
def upload_file(file_name: str, purpose: str) -> str:
with open(file_name, "rb") as file_fd:
response = client.files.create(file=file_fd, purpose=purpose)
return response.id
training_file_id = upload_file(training_file_name, "fine-tune")
validation_file_id = upload_file(validation_file_name, "fine-tune")
print("Training file ID:", training_file_id)
print("Validation file ID:", validation_file_id)Training file ID: file-3wfAfDoYcGrSpaE17qK0vXT0 Validation file ID: file-HhFhnyGJhazYdPcd3wrtvIoX
现在我们可以使用生成的文件和一个可选的模型标识后缀来创建我们的微调任务。响应将包含一个id，您可以用它来获取任务的更新状态。
注意：文件需要先由我们的系统处理，因此您可能会遇到File not ready错误。如果出现这种情况，只需稍等几分钟后重试即可。
MODEL = "gpt-4o-mini-2024-07-18"
response = client.fine_tuning.jobs.create(
training_file=training_file_id,
validation_file=validation_file_id,
model=MODEL,
suffix="recipe-ner",
)
job_id = response.id
print("Job ID:", response.id)
print("Status:", response.status)Job ID: ftjob-UiaiLwGdGBfdLQDBAoQheufN Status: validating_files
你可以向https://api.openai.com/v1/alpha/fine-tunes端点发送GET请求来列出你的alpha微调任务。在这个例子中，你需要确认从上一步获取的ID最终状态为status: succeeded。
完成后，您可以使用result_files从验证集（如果您上传了一个）中采样结果，并使用fine_tuned_model参数中的ID来调用您训练好的模型。
response = client.fine_tuning.jobs.retrieve(job_id)
print("Job ID:", response.id)
print("Status:", response.status)
print("Trained Tokens:", response.trained_tokens)Job ID: ftjob-UiaiLwGdGBfdLQDBAoQheufN Status: running Trained Tokens: None
我们可以通过事件端点跟踪微调的进度。您可以多次重新运行下面的单元格，直到微调完成。
response = client.fine_tuning.jobs.list_events(job_id)
events = response.data
events.reverse()
for event in events:
print(event.message)Step 288/303: training loss=0.00 Step 289/303: training loss=0.01 Step 290/303: training loss=0.00, validation loss=0.31 Step 291/303: training loss=0.00 Step 292/303: training loss=0.00 Step 293/303: training loss=0.00 Step 294/303: training loss=0.00 Step 295/303: training loss=0.00 Step 296/303: training loss=0.00 Step 297/303: training loss=0.00 Step 298/303: training loss=0.01 Step 299/303: training loss=0.00 Step 300/303: training loss=0.00, validation loss=0.04 Step 301/303: training loss=0.16 Step 302/303: training loss=0.00 Step 303/303: training loss=0.00, full validation loss=0.33 Checkpoint created at step 101 with Snapshot ID: ft:gpt-4o-mini-2024-07-18:openai-gtm:recipe-ner:9o1eNlSa:ckpt-step-101 Checkpoint created at step 202 with Snapshot ID: ft:gpt-4o-mini-2024-07-18:openai-gtm:recipe-ner:9o1eNFnj:ckpt-step-202 New fine-tuned model created: ft:gpt-4o-mini-2024-07-18:openai-gtm:recipe-ner:9o1eNNKO The job has successfully completed
现在任务已完成，我们可以从作业中获取微调后的模型ID：
response = client.fine_tuning.jobs.retrieve(job_id)
fine_tuned_model_id = response.fine_tuned_model
if fine_tuned_model_id is None:
raise RuntimeError(
"Fine-tuned model ID not found. Your job has likely not been completed yet."
)
print("Fine-tuned model ID:", fine_tuned_model_id)Fine-tuned model ID: ft:gpt-4o-mini-2024-07-18:openai-gtm:recipe-ner:9o1eNNKO
最后一步是使用您微调后的模型进行推理。与经典的FineTuning类似，您只需调用ChatCompletions并将您新微调的模型名称填入model参数即可。
test_df = recipe_df.loc[201:300]
test_row = test_df.iloc[0]
test_messages = []
test_messages.append({"role": "system", "content": system_message})
user_message = create_user_message(test_row)
test_messages.append({"role": "user", "content": user_message})
pprint(test_messages)[{'content': 'You are a helpful recipe assistant. You are to extract the '
'generic ingredients from each of the recipes provided.',
'role': 'system'},
{'content': 'Title: Beef Brisket\n'
'\n'
'Ingredients: ["4 lb. beef brisket", "1 c. catsup", "1 c. water", '
'"1/2 onion, minced", "2 Tbsp. cider vinegar", "1 Tbsp. prepared '
'horseradish", "1 Tbsp. prepared mustard", "1 tsp. salt", "1/2 '
'tsp. pepper"]\n'
'\n'
'Generic ingredients: ',
'role': 'user'}]
response = client.chat.completions.create(
model=fine_tuned_model_id, messages=test_messages, temperature=0, max_tokens=500
)
print(response.choices[0].message.content)["beef brisket", "catsup", "water", "onion", "cider vinegar", "horseradish", "mustard", "salt", "pepper"]
恭喜，您现在已准备好使用ChatCompletion格式微调自己的模型了！我们期待看到您构建的作品