使用Adala掌握数学推理

import pandas as pd
from datasets import load_dataset
gsm8k = load_dataset("gsm8k", "main")


df_train = pd.DataFrame({'question': gsm8k['train']['question'], 'answer': gsm8k['train']['answer']})
df_test = pd.DataFrame({'question': gsm8k['test']['question'], 'answer': gsm8k['test']['answer']})


df_train.head()

skills = LinearSkillSet(skills=[
   TransformSkill(
       name='math_solver',
       # instructions=prompt
       # Our agent will learn the instructions 
       instructions='',
       # instructions=prompt,
       input_template='Q: {question}',
       # here is the baseline established in Kojima et al., 2022 paper
       # output_template='A: Let’s think step by step. {rationale}\nFinal numerical answer:{answer}',
       output_template='A: {answer}',
       instructions_first=False
   )
])

agent = Agent(
   skills=skills,
  
   # this is where agent receives the ground truth signal
   environment = StaticEnvironment(
       df=df_train,
       matching_function=evaluate_answers
   ),
  
   teacher_runtimes={'gpt4': OpenAIChatRuntime(model='gpt-4-1106-preview')},
   default_teacher_runtime='gpt4'
  
)

# run baseline agent on a test dataset
result_baseline = agent.run(df_test.drop(columns='answer'))


accuracy = StaticEnvironment(df=df_test, \ 
matching_function=evaluate_answers).get_feedback(skills, \ 
result_baseline).get_accuracy()


print(f'Baseline accuracy: {accuracy["answer"]}')

Applying skill: math_solver
100%|██████████| 1319/1319
# Baseline accuracy: 0.4488248673237301

agent.learn(batch_size=5, learning_iterations=5)

print(agent.skills['math_solver'].instructions)


For each arithmetic word problem provided, follow these steps:
- Begin by comprehensively identifying and understanding the relationships, conditions, and any given numerical 
information, including the units of measurement (e.g., days, weeks, months) and the related arithmetic operations 
involved in the scenario.
- Clarify the units and timing of events in the word problem, and restate them if necessary to align with the final
units desired for the solution.
- Select the appropriate mathematical operations necessary to obtain the solution. These operations may involve 
addition, subtraction, multiplication, division, or the application of ratios, percentages, and proportions. Pay 
special attention to any transformations or calculations that entail changes over time, such as growth rates, 
aging, or accumulation of resources, ensuring proper application of increases, decreases, or proportional changes.
- Perform the calculations methodically, verifying at each juncture that the operations and results conform to the 
problem's conditions. Clearly articulate each step of your calculation process, demonstrating the setup of 
equations or proportions and explaining the choice of mathematical methods to maintain clarity and logic throughout
your reasoning.
- Examine each intermediate result within multi-step problems, confirming accuracy before proceeding further. 
Reiterate the logical sequence of steps, especially when the problem involves cumulative or sequential changes, to 
ensure continuity and logical progression in the calculation process.
- Conclude by cross-checking the final answer against the conditions and expectations set forth by the problem. 
Resolve any inconsistencies encountered and articulate a coherent, exact solution.

result_new = agent.run(df_test.drop(columns='answer'))


accuracy = StaticEnvironment(df=df_test, \ 
matching_function=evaluate_answers).get_feedback(skills, \ 
result_baseline).get_accuracy()


print(f'New accuracy: {accuracy["answer"]}')

New accuracy: 0.7414708112206216