减少计算负载的策略¶
本教程涵盖了两种策略，用于修剪TPOT的计算负载以减少运行时间。
连续减半¶
这个想法最初由Parmentier等人在"TPOT-SH: a Faster Optimization Algorithm to Solve the AutoML Problem on Large Datasets"中使用TPOT进行了测试。该算法分为两个阶段运行。最初，它使用一个小数据子集和大量种群规模来训练早期世代。随后的世代则在更大甚至完整的数据部分上评估一组较小的有前景的管道。这种方法通过初步的粗略评估，随后进行更全面的评估，快速识别出表现最佳的管道配置。更多关于此策略的信息请参见教程8。
在本教程中，我们将涵盖以下参数：
population_size
initial_population_size
population_scaling
generations_until_end_population
budget_range
generations_until_end_budget
budget_scaling
stepwise_steps
种群大小是每代评估的个体数量。预算指的是采样的数据比例。通过调整这些参数，我们可以控制预算增加的速度以及种群大小随时间的变化。通常情况下，这将用于启动算法，通过在数据的较小子集上评估大量管道来快速缩小当前最佳模型的范围，然后在较少的数据集上使用更大的样本获得更好的估计。这可以通过不花费太多时间评估表现不佳的管道来降低总体计算成本。
population_size 决定了每一代要评估的个体数量。有时我们可能希望在早期世代评估更多或更少的个体。initial_population_size 参数指定了种群的初始大小。种群大小将在 generations_until_end_population 世代的过程中逐渐从 initial_population_size 过渡到 population_size。population_scaling 决定了这种过渡的速度。在 generations_until_end_population 世代中的插值是逐步进行的，步数由 stepwise_steps 指定。
预算缩放的过程也是如此。
以下单元格展示了在给定设置下，种群规模和预算如何随时间变化。（请注意，tpot 在这个数据集上恰好收敛得相当快，但我们关闭了早期停止以获得完整的运行。）
In [ ]:
Copied!
import matplotlib.pyplot as plt
import tpot2
population_size=30
initial_population_size=100
population_scaling = .5
generations_until_end_population = 50
budget_range = [.3,1]
generations_until_end_budget=50
budget_scaling = .5
stepwise_steps = 5
#Population and budget use stepwise
fig, ax1 = plt.subplots()
ax2 = ax1.twinx()
interpolated_values_population = tpot2.utils.beta_interpolation(start=initial_population_size, end=population_size, n=generations_until_end_population, n_steps=stepwise_steps, scale=population_scaling)
interpolated_values_budget = tpot2.utils.beta_interpolation(start=budget_range[0], end=budget_range[1], n=generations_until_end_budget, n_steps=stepwise_steps, scale=budget_scaling)
ax1.step(list(range(len(interpolated_values_population))), interpolated_values_population, label=f"population size")
ax2.step(list(range(len(interpolated_values_budget))), interpolated_values_budget, label=f"budget", color='r')
ax1.set_xlabel("generation")
ax1.set_ylabel("population size")
ax2.set_ylabel("bugdet")
ax1.legend(loc='center left', bbox_to_anchor=(1.1, 0.4))
ax2.legend(loc='center left', bbox_to_anchor=(1.1, 0.3))
plt.show()
import matplotlib.pyplot as plt
import tpot2
population_size=30
initial_population_size=100
population_scaling = .5
generations_until_end_population = 50
budget_range = [.3,1]
generations_until_end_budget=50
budget_scaling = .5
stepwise_steps = 5
#Population and budget use stepwise
fig, ax1 = plt.subplots()
ax2 = ax1.twinx()
interpolated_values_population = tpot2.utils.beta_interpolation(start=initial_population_size, end=population_size, n=generations_until_end_population, n_steps=stepwise_steps, scale=population_scaling)
interpolated_values_budget = tpot2.utils.beta_interpolation(start=budget_range[0], end=budget_range[1], n=generations_until_end_budget, n_steps=stepwise_steps, scale=budget_scaling)
ax1.step(list(range(len(interpolated_values_population))), interpolated_values_population, label=f"population size")
ax2.step(list(range(len(interpolated_values_budget))), interpolated_values_budget, label=f"budget", color='r')
ax1.set_xlabel("generation")
ax1.set_ylabel("population size")
ax2.set_ylabel("bugdet")
ax1.legend(loc='center left', bbox_to_anchor=(1.1, 0.4))
ax2.legend(loc='center left', bbox_to_anchor=(1.1, 0.3))
plt.show()
In [ ]:
Copied!
# A Graph pipeline starting with at least one selector as a leaf, potentially followed by a series
# of stacking classifiers or transformers, and ending with a classifier. The graph will have at most 15 nodes and a max depth of 6.
import tpot2
import sklearn
import sklearn.datasets
import numpy as np
import time
import tpot2
import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression
import sklearn
X, y = sklearn.datasets.load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, random_state=1)
scorer = sklearn.metrics.make_scorer(sklearn.metrics.roc_auc_score, needs_proba=True, multi_class='ovr')
est = tpot2.TPOTEstimator(
generations=50,
max_time_mins=None,
scorers=['roc_auc_ovr'],
scorers_weights=[1],
classification=True,
search_space = 'linear',
n_jobs=32,
cv=10,
verbose=3,
population_size=population_size,
initial_population_size=initial_population_size,
population_scaling = population_scaling,
generations_until_end_population = generations_until_end_population,
budget_range = budget_range,
generations_until_end_budget=generations_until_end_budget,
)
start = time.time()
est.fit(X_train, y_train)
print(f"total time: {time.time()-start}")
print("test score: ", scorer(est, X_test, y_test))
# A Graph pipeline starting with at least one selector as a leaf, potentially followed by a series
# of stacking classifiers or transformers, and ending with a classifier. The graph will have at most 15 nodes and a max depth of 6.
import tpot2
import sklearn
import sklearn.datasets
import numpy as np
import time
import tpot2
import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression
import sklearn
X, y = sklearn.datasets.load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, random_state=1)
scorer = sklearn.metrics.make_scorer(sklearn.metrics.roc_auc_score, needs_proba=True, multi_class='ovr')
est = tpot2.TPOTEstimator(
generations=50,
max_time_mins=None,
scorers=['roc_auc_ovr'],
scorers_weights=[1],
classification=True,
search_space = 'linear',
n_jobs=32,
cv=10,
verbose=3,
population_size=population_size,
initial_population_size=initial_population_size,
population_scaling = population_scaling,
generations_until_end_population = generations_until_end_population,
budget_range = budget_range,
generations_until_end_budget=generations_until_end_budget,
)
start = time.time()
est.fit(X_train, y_train)
print(f"total time: {time.time()-start}")
print("test score: ", scorer(est, X_test, y_test))
CV 早期剪枝¶
大多数情况下，我们将使用交叉验证来评估管道。然而，我们通常可以在前几次折叠中判断出管道是否有合理的可能性超越之前的最佳管道。例如，如果到目前为止的最佳得分是0.92 AUROC，而我们当前管道的前五次折叠的平均得分仅为0.61左右，我们可以合理地相信接下来的五次折叠不太可能使这个管道领先于其他管道。通过不计算剩余的折叠，我们可以节省大量的计算资源。TPOT可以使用两种策略来实现这一点（更多关于这些策略的信息请参见教程8）。
- 阈值剪枝：管道必须在每个交叉验证（CV）折叠中达到预定义的百分位阈值（基于之前的管道分数）才能继续。
- 选择剪枝：在每个种群中，只有前N%的管道（根据前一个交叉验证折的性能排名）被选择在下一个折中进行评估。
如果前几次交叉验证分数不理想，我们可以通过提前终止单个管道的评估来进一步减少计算负担。请注意，这与整个算法的提前停止不同。在本节中，我们将介绍：
threshold_evaluation_pruning
threshold_evaluation_scaling
min_history_threshold
selection_evaluation_pruning
selection_evaluation_scaling
阈值早停使用之前的分数来识别并终止表现不佳的管道的交叉验证评估。我们从之前评估的管道中计算百分位数分数。管道必须在每个折叠中达到给定的百分位数才能进行下一次评估，否则管道将被丢弃。
threshold_evaluation_pruning 参数是一个列表，指定了用于评估早期停止的起始和结束百分位数。W threshold_evaluation_scaling 参数是一个浮点数，控制阈值从起始百分位数移动到结束百分位数的速率。min_history_threshold 参数指定了在使用阈值早期停止之前所需的最小历史分数数量。这确保了算法有足够的历史数据来做出何时停止评估管道的明智决策。
选择早期停止在每次折叠后使用选择算法来选择哪些算法将在下一次折叠中进行评估。例如，在折叠1上评估了100个个体后，我们可能只想在剩余的折叠中评估最好的50个。
selection_evaluation_pruning 参数是一个列表，指定了每轮交叉验证中选择的种群大小的下限和上限百分比。这用于确定下一代中要评估的个体。selection_evaluation_scaling 参数是一个浮点数，控制选择阈值从起始百分位数移动到结束百分位数的速率。
通过操作这些参数，我们可以控制算法如何选择个体以评估下一代，以及何时停止评估表现不佳的管道。
实际上，这些参数的值将取决于具体问题和可用的计算资源。
在接下来的部分中，我们将向您展示如何在Jupyter Notebook中使用Python代码设置和操作这些参数。我们还将提供这些参数如何影响算法性能的示例。
（请注意，在这些小型测试案例中，您可能不会注意到太多或任何性能改进，这些改进在具有更大数据集和较慢评估管道的实际场景中可能更有益。）
注意事项： 了解CV剪枝如何与进化算法相互作用非常重要。当使用这些方法之一剪枝管道时，它们将从活跃种群中移除，因此不再用于通知TPOT算法。如果剪枝过多的管道，这可能会减少每代管道的多样性，并限制TPOT的学习能力。此外，剪枝方法可能会影响TPOT运行所需的时间。如果剪枝算法移除了性能稍差但运行速度较快的管道，TPOT很可能会在下一代中仅填充运行速度较慢的管道，从而在技术上增加总运行时间。这可能是可以接受的，因为更多的计算资源被用于性能更高的管道。
In [1]:
Copied!
import matplotlib.pyplot as plt
import tpot2
import time
import sklearn
import sklearn.datasets
threshold_evaluation_pruning = [30, 90]
threshold_evaluation_scaling = .2 #.5
cv = 10
#Population and budget use stepwise
fig, ax1 = plt.subplots()
interpolated_values = tpot2.utils.beta_interpolation(start=threshold_evaluation_pruning[0], end=threshold_evaluation_pruning[-1], n=cv, n_steps=cv, scale=threshold_evaluation_scaling)
ax1.step(list(range(len(interpolated_values))), interpolated_values, label=f"threshold")
ax1.set_xlabel("fold")
ax1.set_ylabel("percentile")
#ax1.legend(loc='center left', bbox_to_anchor=(1.1, 0.4))
plt.show()
import matplotlib.pyplot as plt
import tpot2
import time
import sklearn
import sklearn.datasets
threshold_evaluation_pruning = [30, 90]
threshold_evaluation_scaling = .2 #.5
cv = 10
#Population and budget use stepwise
fig, ax1 = plt.subplots()
interpolated_values = tpot2.utils.beta_interpolation(start=threshold_evaluation_pruning[0], end=threshold_evaluation_pruning[-1], n=cv, n_steps=cv, scale=threshold_evaluation_scaling)
ax1.step(list(range(len(interpolated_values))), interpolated_values, label=f"threshold")
ax1.set_xlabel("fold")
ax1.set_ylabel("percentile")
#ax1.legend(loc='center left', bbox_to_anchor=(1.1, 0.4))
plt.show()
In [2]:
Copied!
import tpot2
from tpot2.search_spaces.pipelines import *
from tpot2.search_spaces.nodes import *
from tpot2.config.get_configspace import get_search_space
import sklearn.model_selection
import sklearn
selectors = get_search_space(["selectors","selectors_classification", "Passthrough"], random_state=42,)
estimators = get_search_space(['XGBClassifier'],random_state=42,)
scalers = get_search_space(["scalers","Passthrough"],random_state=42,)
transformers_layer =UnionPipeline([
ChoicePipeline([
DynamicUnionPipeline(get_search_space(["transformers"], random_state=42,)),
get_search_space("SkipTransformer"),
]),
get_search_space("Passthrough")
]
)
search_space = SequentialPipeline(search_spaces=[
scalers,
selectors,
transformers_layer,
estimators,
])
import tpot2
from tpot2.search_spaces.pipelines import *
from tpot2.search_spaces.nodes import *
from tpot2.config.get_configspace import get_search_space
import sklearn.model_selection
import sklearn
selectors = get_search_space(["selectors","selectors_classification", "Passthrough"], random_state=42,)
estimators = get_search_space(['XGBClassifier'],random_state=42,)
scalers = get_search_space(["scalers","Passthrough"],random_state=42,)
transformers_layer =UnionPipeline([
ChoicePipeline([
DynamicUnionPipeline(get_search_space(["transformers"], random_state=42,)),
get_search_space("SkipTransformer"),
]),
get_search_space("Passthrough")
]
)
search_space = SequentialPipeline(search_spaces=[
scalers,
selectors,
transformers_layer,
estimators,
])
In [3]:
Copied!
import matplotlib.pyplot as plt
import tpot2
import time
import sklearn
import sklearn.datasets
scorer = sklearn.metrics.make_scorer(sklearn.metrics.roc_auc_score, needs_proba=True, multi_class='ovr')
X, y = sklearn.datasets.make_classification(n_samples=5000, n_features=20, n_classes=5, random_state=1, n_informative=15, n_redundant=5, n_repeated=0, n_clusters_per_class=3, class_sep=.8)
X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, random_state=1)
# search_space = tpot2.config.template_search_spaces.get_template_search_spaces("linear",inner_predictors=False, random_state=42)
import matplotlib.pyplot as plt
import tpot2
import time
import sklearn
import sklearn.datasets
scorer = sklearn.metrics.make_scorer(sklearn.metrics.roc_auc_score, needs_proba=True, multi_class='ovr')
X, y = sklearn.datasets.make_classification(n_samples=5000, n_features=20, n_classes=5, random_state=1, n_informative=15, n_redundant=5, n_repeated=0, n_clusters_per_class=3, class_sep=.8)
X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, random_state=1)
# search_space = tpot2.config.template_search_spaces.get_template_search_spaces("linear",inner_predictors=False, random_state=42)
/home/ribeirop/common/miniconda3/envs/tpot2env/lib/python3.10/site-packages/sklearn/metrics/_scorer.py:548: FutureWarning: The `needs_threshold` and `needs_proba` parameter are deprecated in version 1.4 and will be removed in 1.6. You can either let `response_method` be `None` or set it to `predict` to preserve the same behaviour. warnings.warn(
In [4]:
Copied!
# no pruning
est = tpot2.TPOTEstimator(
generations=10,
max_time_mins=None,
scorers=['roc_auc_ovr'],
scorers_weights=[1],
classification=True,
search_space = search_space,
population_size=100,
n_jobs=32,
cv=cv,
verbose=3,
random_state=42,
)
start = time.time()
est.fit(X_train, y_train)
print(f"total time: {time.time()-start}")
print("test score: ", scorer(est, X_test, y_test))
# no pruning
est = tpot2.TPOTEstimator(
generations=10,
max_time_mins=None,
scorers=['roc_auc_ovr'],
scorers_weights=[1],
classification=True,
search_space = search_space,
population_size=100,
n_jobs=32,
cv=cv,
verbose=3,
random_state=42,
)
start = time.time()
est.fit(X_train, y_train)
print(f"total time: {time.time()-start}")
print("test score: ", scorer(est, X_test, y_test))
Generation: 10%|█ | 1/10 [03:02<27:26, 182.98s/it]
Generation: 1 Best roc_auc_score score: 0.915278983783422
Generation: 20%|██ | 2/10 [06:11<24:51, 186.47s/it]
Generation: 2 Best roc_auc_score score: 0.9253965903409787
Generation: 30%|███ | 3/10 [10:33<25:46, 220.92s/it]
Generation: 3 Best roc_auc_score score: 0.9340480147661712
Generation: 40%|████ | 4/10 [15:07<24:10, 241.71s/it]
Generation: 4 Best roc_auc_score score: 0.9340480147661712
Generation: 50%|█████ | 5/10 [21:46<24:52, 298.58s/it]
Generation: 5 Best roc_auc_score score: 0.9340480147661712
Generation: 60%|██████ | 6/10 [25:45<18:32, 278.19s/it]
Generation: 6 Best roc_auc_score score: 0.9340480147661712
Generation: 70%|███████ | 7/10 [29:00<12:32, 250.97s/it]
Generation: 7 Best roc_auc_score score: 0.9340480147661712
Generation: 80%|████████ | 8/10 [34:14<09:02, 271.09s/it]
Generation: 8 Best roc_auc_score score: 0.9349103682633716
Generation: 90%|█████████ | 9/10 [37:44<04:12, 252.09s/it]
Generation: 9 Best roc_auc_score score: 0.9372520424560744
Generation: 100%|██████████| 10/10 [43:09<00:00, 258.96s/it]
Generation: 10 Best roc_auc_score score: 0.9398288783489072
total time: 2602.0502972602844 test score: 0.9426160568071598
In [5]:
Copied!
import tpot2.config
import tpot2.config.template_search_spaces
import tpot2.search_spaces
# search_space = tpot2.config.get_search_space(["RandomForestClassifier"])
est = tpot2.TPOTEstimator(
generations=10,
max_time_mins=None,
scorers=['roc_auc_ovr'],
scorers_weights=[1],
classification=True,
search_space = search_space,
population_size=100,
n_jobs=32,
cv=cv,
verbose=3,
random_state=42,
threshold_evaluation_pruning = threshold_evaluation_pruning,
threshold_evaluation_scaling = threshold_evaluation_scaling,
)
start = time.time()
est.fit(X_train, y_train)
print(f"total time: {time.time()-start}")
print("test score: ", scorer(est, X_test, y_test))
import tpot2.config
import tpot2.config.template_search_spaces
import tpot2.search_spaces
# search_space = tpot2.config.get_search_space(["RandomForestClassifier"])
est = tpot2.TPOTEstimator(
generations=10,
max_time_mins=None,
scorers=['roc_auc_ovr'],
scorers_weights=[1],
classification=True,
search_space = search_space,
population_size=100,
n_jobs=32,
cv=cv,
verbose=3,
random_state=42,
threshold_evaluation_pruning = threshold_evaluation_pruning,
threshold_evaluation_scaling = threshold_evaluation_scaling,
)
start = time.time()
est.fit(X_train, y_train)
print(f"total time: {time.time()-start}")
print("test score: ", scorer(est, X_test, y_test))
Generation: 10%|█ | 1/10 [03:37<32:37, 217.51s/it]
Generation: 1 Best roc_auc_score score: 0.915278983783422
Generation: 20%|██ | 2/10 [05:38<21:26, 160.86s/it]
Generation: 2 Best roc_auc_score score: 0.915278983783422
Generation: 30%|███ | 3/10 [08:28<19:14, 164.99s/it]
Generation: 3 Best roc_auc_score score: 0.9212056169353746
Generation: 40%|████ | 4/10 [10:14<14:09, 141.53s/it]
Generation: 4 Best roc_auc_score score: 0.9212056169353746
Generation: 50%|█████ | 5/10 [13:23<13:13, 158.71s/it]
Generation: 5 Best roc_auc_score score: 0.9212056169353746
Generation: 60%|██████ | 6/10 [16:59<11:52, 178.22s/it]
Generation: 6 Best roc_auc_score score: 0.9212056169353746
Generation: 70%|███████ | 7/10 [19:54<08:51, 177.14s/it]
Generation: 7 Best roc_auc_score score: 0.9212056169353746
Generation: 80%|████████ | 8/10 [23:15<06:09, 184.68s/it]
Generation: 8 Best roc_auc_score score: 0.9277731650225494
Generation: 90%|█████████ | 9/10 [26:06<03:00, 180.55s/it]
Generation: 9 Best roc_auc_score score: 0.930278306286007
Generation: 100%|██████████| 10/10 [28:29<00:00, 170.96s/it]
Generation: 10 Best roc_auc_score score: 0.9336783524637781
/home/ribeirop/common/miniconda3/envs/tpot2env/lib/python3.10/site-packages/sklearn/decomposition/_fastica.py:595: UserWarning: n_components is too large: it will be set to 16 warnings.warn( /home/ribeirop/common/miniconda3/envs/tpot2env/lib/python3.10/site-packages/sklearn/decomposition/_fastica.py:128: ConvergenceWarning: FastICA did not converge. Consider increasing tolerance or the maximum number of iterations. warnings.warn(
total time: 1717.3964531421661 test score: 0.9449535389019192
In [6]:
Copied!
import matplotlib.pyplot as plt
import tpot2
selection_evaluation_pruning = [.9, .3]
selection_evaluation_scaling = .2
#Population and budget use stepwise
fig, ax1 = plt.subplots()
interpolated_values = tpot2.utils.beta_interpolation(start=selection_evaluation_pruning[0], end=selection_evaluation_pruning[-1], n=cv, n_steps=cv, scale=selection_evaluation_scaling)
ax1.step(list(range(len(interpolated_values))), interpolated_values, label=f"threshold")
ax1.set_xlabel("fold")
ax1.set_ylabel("percent to select")
#ax1.legend(loc='center left', bbox_to_anchor=(1.1, 0.4))
plt.show()
import matplotlib.pyplot as plt
import tpot2
selection_evaluation_pruning = [.9, .3]
selection_evaluation_scaling = .2
#Population and budget use stepwise
fig, ax1 = plt.subplots()
interpolated_values = tpot2.utils.beta_interpolation(start=selection_evaluation_pruning[0], end=selection_evaluation_pruning[-1], n=cv, n_steps=cv, scale=selection_evaluation_scaling)
ax1.step(list(range(len(interpolated_values))), interpolated_values, label=f"threshold")
ax1.set_xlabel("fold")
ax1.set_ylabel("percent to select")
#ax1.legend(loc='center left', bbox_to_anchor=(1.1, 0.4))
plt.show()
In [7]:
Copied!
est = tpot2.TPOTEstimator(
generations=10,
max_time_mins=None,
scorers=['roc_auc_ovr'],
scorers_weights=[1],
classification=True,
search_space = search_space,
population_size=100,
n_jobs=32,
cv=cv,
verbose=3,
random_state=42,
selection_evaluation_pruning = selection_evaluation_pruning,
selection_evaluation_scaling = selection_evaluation_scaling,
)
start = time.time()
est.fit(X_train, y_train)
print(f"total time: {time.time()-start}")
print("test score: ", scorer(est, X_test, y_test))
est = tpot2.TPOTEstimator(
generations=10,
max_time_mins=None,
scorers=['roc_auc_ovr'],
scorers_weights=[1],
classification=True,
search_space = search_space,
population_size=100,
n_jobs=32,
cv=cv,
verbose=3,
random_state=42,
selection_evaluation_pruning = selection_evaluation_pruning,
selection_evaluation_scaling = selection_evaluation_scaling,
)
start = time.time()
est.fit(X_train, y_train)
print(f"total time: {time.time()-start}")
print("test score: ", scorer(est, X_test, y_test))
Generation: 10%|█ | 1/10 [03:28<31:13, 208.18s/it]
Generation: 1 Best roc_auc_score score: 0.915278983783422
Generation: 20%|██ | 2/10 [05:29<20:54, 156.85s/it]
Generation: 2 Best roc_auc_score score: 0.9169454884359916
Generation: 30%|███ | 3/10 [08:32<19:42, 168.98s/it]
Generation: 3 Best roc_auc_score score: 0.9176524001433647
Generation: 40%|████ | 4/10 [11:08<16:22, 163.77s/it]
Generation: 4 Best roc_auc_score score: 0.9176524001433647
Generation: 50%|█████ | 5/10 [15:05<15:51, 190.38s/it]
Generation: 5 Best roc_auc_score score: 0.9176524001433647
Generation: 60%|██████ | 6/10 [18:02<12:23, 185.84s/it]
Generation: 6 Best roc_auc_score score: 0.9206270411396777
Generation: 70%|███████ | 7/10 [21:12<09:20, 186.94s/it]
Generation: 7 Best roc_auc_score score: 0.9224227652034017
Generation: 80%|████████ | 8/10 [23:53<05:57, 178.80s/it]
Generation: 8 Best roc_auc_score score: 0.9224227652034017
Generation: 90%|█████████ | 9/10 [26:37<02:54, 174.24s/it]
Generation: 9 Best roc_auc_score score: 0.9224227652034017
Generation: 100%|██████████| 10/10 [29:21<00:00, 176.14s/it]
Generation: 10 Best roc_auc_score score: 0.9224227652034017
/home/ribeirop/common/miniconda3/envs/tpot2env/lib/python3.10/site-packages/sklearn/decomposition/_fastica.py:595: UserWarning: n_components is too large: it will be set to 20 warnings.warn(
total time: 1777.245548248291 test score: 0.9253163988063096
In [8]:
Copied!
est.evaluated_individuals[est.evaluated_individuals['roc_auc_score_step_9']>0]
est.evaluated_individuals[est.evaluated_individuals['roc_auc_score_step_9']>0]
Out[8]:
| roc_auc_score | 父母 | 变异函数 | 个体 | 世代 | roc_auc_score_step_0 | 提交时间戳 | 完成时间戳 | 评估错误 | roc_auc_score_step_1 | roc_auc_score_step_2 | roc_auc_score_step_3 | roc_auc_score_step_4 | roc_auc_score_step_5 | roc_auc_score_step_6 | roc_auc_score_step_7 | roc_auc_score_step_8 | roc_auc_score_step_9 | 帕累托前沿 | 实例 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 0.848068 | NaN | NaN | | 0.0 |
0.846478 |
1.727821e+09 |
1.727821e+09 |
None |
0.839894 |
0.844619 |
0.848321 |
0.846915 |
0.857902 |
0.855875 |
0.827655 |
0.850938 |
0.862081 |
NaN |
(Passthrough(), RFE(estimator=ExtraTreesClassi... |
|
| 4 | 0.831502 | NaN | NaN | | 0.0 |
0.817219 |
1.727822e+09 |
1.727822e+09 |
None |
0.827888 |
0.821911 |
0.825558 |
0.830020 |
0.831529 |
0.836955 |
0.844634 |
0.832499 |
0.846805 |
NaN |
(StandardScaler(), VarianceThreshold(threshold... |
|
| 5 | 0.830374 | NaN | NaN | | 0.0 |
0.817150 |
1.727822e+09 |
1.727822e+09 |
None |
0.831885 |
0.820694 |
0.824899 |
0.824409 |
0.827861 |
0.833923 |
0.844308 |
0.832798 |
0.845818 |
NaN |
(MinMaxScaler(), SelectFromModel(estimator=Ext... |
|
| 6 | 0.850091 | NaN | NaN | | 0.0 |
0.843524 |
1.727821e+09 |
1.727821e+09 |
None |
0.841176 |
0.840619 |
0.846209 |
0.849561 |
0.854367 |
0.858035 |
0.860165 |
0.845179 |
0.862077 |
NaN |
(Normalizer(norm='max'), SelectFwe(alpha=0.000... |
|
| 9 | 0.855569 | NaN | NaN | | 0.0 |
0.847828 |
1.727821e+09 |
1.727821e+09 |
None |
0.846977 |
0.849937 |
0.853201 |
0.857401 |
0.859119 |
0.857783 |
0.863300 |
0.851526 |
0.868619 |
NaN |
(Normalizer(norm='l1'), Passthrough(), Feature... |
|
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 990 | 0.821990 | (742, 742) | ind_mutate | | 9.0 |
0.813408 |
1.727823e+09 |
1.727823e+09 |
None |
0.815070 |
0.830230 |
0.823377 |
0.823119 |
0.831668 |
0.827358 |
0.817293 |
0.814207 |
0.824168 |
NaN |
(MinMaxScaler(), SelectPercentile(percentile=5... |
|
| 991 | 0.899339 | (100, 100) | ind_mutate | | 9.0 |
0.893247 |
1.727823e+09 |
1.727823e+09 |
None |
0.903268 |
0.894318 |
0.890992 |
0.902956 |
0.902985 |
0.898020 |
0.904124 |
0.898141 |
0.905341 |
NaN |
(Normalizer(norm='l1'), SelectFwe(alpha=0.0034... |
|
| 992 | 0.870868 | (179, 14) | ind_crossover | | 9.0 |
0.871226 |
1.727823e+09 |
1.727823e+09 |
None |
0.854742 |
0.865197 |
0.872427 |
0.869312 |
0.880744 |
0.872265 |
0.877524 |
0.866365 |
0.878881 |
NaN |
(Normalizer(norm='l1'), SelectFromModel(estima... |
|
| 994 | 0.815212 | (362, 362) | ind_mutate | | 9.0 |
0.830802 |
1.727823e+09 |
1.727823e+09 |
None |
0.807573 |
0.817124 |
0.800161 |
0.819611 |
0.818811 |
0.833061 |
0.803679 |
0.794094 |
0.827200 |
NaN |
(Normalizer(norm='l1'), SelectPercentile(perce... |
|
| 995 | 0.865588 | (670, 670) | ind_mutate | | 9.0 |
0.867900 |
1.727823e+09 |
1.727823e+09 |
None |
0.870824 |
0.876060 |
0.871468 |
0.853827 |
0.864080 |
0.867749 |
0.853553 |
0.849620 |
0.880796 |
NaN |
(Normalizer(), SelectPercentile(percentile=71.... |
|
324 行 × 20 列
以上所有方法可以独立使用或同时使用，如下所示：
In [ ]:
Copied!
est = tpot2.TPOTEstimator(
generations=10,
max_time_mins=None,
scorers=['roc_auc_ovr'],
scorers_weights=[1],
classification=True,
search_space = search_space,
population_size=30,
n_jobs=3,
cv=cv,
verbose=3,
population_size=population_size,
initial_population_size=initial_population_size,
population_scaling = population_scaling,
generations_until_end_population = generations_until_end_population,
budget_range = budget_range,
generations_until_end_budget=generations_until_end_budget,
threshold_evaluation_pruning = threshold_evaluation_pruning,
threshold_evaluation_scaling = threshold_evaluation_scaling,
selection_evaluation_pruning = selection_evaluation_pruning,
selection_evaluation_scaling = selection_evaluation_scaling,
)
start = time.time()
est.fit(X_train, y_train)
print(f"total time: {time.time()-start}")
print("test score: ", scorer(est, X_test, y_test))
est = tpot2.TPOTEstimator(
generations=10,
max_time_mins=None,
scorers=['roc_auc_ovr'],
scorers_weights=[1],
classification=True,
search_space = search_space,
population_size=30,
n_jobs=3,
cv=cv,
verbose=3,
population_size=population_size,
initial_population_size=initial_population_size,
population_scaling = population_scaling,
generations_until_end_population = generations_until_end_population,
budget_range = budget_range,
generations_until_end_budget=generations_until_end_budget,
threshold_evaluation_pruning = threshold_evaluation_pruning,
threshold_evaluation_scaling = threshold_evaluation_scaling,
selection_evaluation_pruning = selection_evaluation_pruning,
selection_evaluation_scaling = selection_evaluation_scaling,
)
start = time.time()
est.fit(X_train, y_train)
print(f"total time: {time.time()-start}")
print("test score: ", scorer(est, X_test, y_test))