介绍¶

TPOT为用户提供了许多自定义搜索空间的选项，从超参数范围到模型选择再到管道配置。TPOT能够选择模型、优化其超参数，并构建复杂的管道结构。每个细节级别都有多个自定义选项。本教程将首先探讨如何为单个方法设置超参数搜索空间。接下来，我们将描述如何同时设置模型选择和超参数调优。最后，我们将介绍如何利用这些步骤为多步骤的固定管道配置搜索空间，以及让TPOT优化管道结构本身。

使用ConfigSpace的超参数搜索空间¶

超参数搜索空间是使用这里找到的ConfigSpace包定义的。关于如何设置超参数空间的更多信息可以在他们的文档这里找到。

TPOT 使用 ConfigSpace.ConfigurationSpace 对象来定义单个模型的超参数搜索空间。该对象可用于跟踪所需的超参数，并提供从该空间随机采样的函数。

简而言之，你可以使用ConfigSpace的Integer、Float和Categorical函数来定义每个参数使用的值范围。或者，可以使用包含(min,max)整数或浮点数的元组来指定整数/浮点数搜索空间，并使用列表来指定分类搜索空间。对于不需要调整的参数，也可以提供一个固定值。ConfigurationSpace的space参数接受一个参数名称到这些范围的字典。

注意：如果您想要可重复的结果，您需要在搜索空间中设置一个固定的random_state。

以下是RandomForest的超参数范围示例

In [1]:

Copied!





from ConfigSpace import ConfigurationSpace
from ConfigSpace import ConfigurationSpace, Integer, Float, Categorical, Normal
from sklearn.ensemble import RandomForestClassifier
import tpot2
import numpy as np
import sklearn
import sklearn.datasets

rf_configspace = ConfigurationSpace(
    space = {
            'n_estimators': 128, #as recommended by Oshiro et al. (2012
            'max_features': Float("max_features", bounds=(0.01,1), log=True), #log scale like autosklearn?
            'criterion': Categorical("criterion", ['gini', 'entropy']),
            'min_samples_split': Integer("min_samples_split", bounds=(2, 20)),
            'min_samples_leaf': Integer("min_samples_leaf", bounds=(1, 20)),
            'bootstrap': Categorical("bootstrap", [True, False]),
            #random_state = 1, # If you want results to be reproducible, you can set a fixed random_state.
        }
)

hyperparameters = dict(rf_configspace.sample_configuration())
print("sampled hyperparameters")
print(hyperparameters)

rf = RandomForestClassifier(**hyperparameters)
rf
from ConfigSpace import ConfigurationSpace
from ConfigSpace import ConfigurationSpace, Integer, Float, Categorical, Normal
from sklearn.ensemble import RandomForestClassifier
import tpot2
import numpy as np
import sklearn
import sklearn.datasets

rf_configspace = ConfigurationSpace(
    space = {
            'n_estimators': 128, #as recommended by Oshiro et al. (2012
            'max_features': Float("max_features", bounds=(0.01,1), log=True), #log scale like autosklearn?
            'criterion': Categorical("criterion", ['gini', 'entropy']),
            'min_samples_split': Integer("min_samples_split", bounds=(2, 20)),
            'min_samples_leaf': Integer("min_samples_leaf", bounds=(1, 20)),
            'bootstrap': Categorical("bootstrap", [True, False]),
            #random_state = 1, # If you want results to be reproducible, you can set a fixed random_state.
        }
)

hyperparameters = dict(rf_configspace.sample_configuration())
print("sampled hyperparameters")
print(hyperparameters)

rf = RandomForestClassifier(**hyperparameters)
rf

sampled hyperparameters
{'bootstrap': False, 'criterion': 'entropy', 'max_features': 0.1574830347299, 'min_samples_leaf': 10, 'min_samples_split': 6, 'n_estimators': 128}

Out[1]:

RandomForestClassifier(bootstrap=False, criterion='entropy',
                       max_features=0.1574830347299, min_samples_leaf=10,
                       min_samples_split=6, n_estimators=128)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

更简单地说：

In [2]:

Copied!





rf_configspace = ConfigurationSpace(
    space = {
            'n_estimators': 128, #as recommended by Oshiro et al. (2012
            'max_features':(0.01,1), #not log scaled
            'criterion': ['gini', 'entropy'],
            'min_samples_split': (2, 20),
            'min_samples_leaf': (1, 20),
            'bootstrap': [True, False],
            #random_state = 1, # If you want results to be reproducible, you can set a fixed random_state.
        }
)

hyperparameters = dict(rf_configspace.sample_configuration())
print("sampled hyperparameters")
print(hyperparameters)

rf = RandomForestClassifier(**hyperparameters)
rf
rf_configspace = ConfigurationSpace(
    space = {
            'n_estimators': 128, #as recommended by Oshiro et al. (2012
            'max_features':(0.01,1), #not log scaled
            'criterion': ['gini', 'entropy'],
            'min_samples_split': (2, 20),
            'min_samples_leaf': (1, 20),
            'bootstrap': [True, False],
            #random_state = 1, # If you want results to be reproducible, you can set a fixed random_state.
        }
)

hyperparameters = dict(rf_configspace.sample_configuration())
print("sampled hyperparameters")
print(hyperparameters)

rf = RandomForestClassifier(**hyperparameters)
rf

sampled hyperparameters
{'bootstrap': True, 'criterion': 'entropy', 'max_features': 0.2601475241557, 'min_samples_leaf': 17, 'min_samples_split': 3, 'n_estimators': 128}

Out[2]:

RandomForestClassifier(criterion='entropy', max_features=0.2601475241557,
                       min_samples_leaf=17, min_samples_split=3,
                       n_estimators=128)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

TPOT 搜索空间¶

TPOT 允许您为单个方法和管道结构创建超参数搜索空间。例如，TPOT 可以创建线性管道、树或图。

TPOT 的搜索空间可以在 search_spaces 模块中找到。主要有两种类型的搜索空间：节点搜索空间和管道搜索空间。节点搜索空间指定了一个单一的 sklearn BaseEstimator 搜索空间。管道搜索空间定义了一组节点搜索空间的可能结构。这些搜索空间接受节点搜索空间，并使用该搜索空间中的节点生成管道。由于 sklearn 管道也是 BaseEstimator，因此管道搜索空间在技术上也是节点搜索空间。这意味着管道搜索空间可以接受其他管道搜索空间，以定义更复杂的结构。节点搜索空间和管道搜索空间之间的主要区别在于，管道搜索空间必须接受另一个搜索空间作为输入，以提供其各个节点。因此，所有搜索空间最终都会在最低级别以节点搜索空间结束。请注意，管道搜索空间的参数可能有所不同，有些只接受单个搜索空间，有些接受列表，有些接受多个定义的参数。

节点搜索空间¶

名称	信息
EstimatorNode	接收一个ConfigSpace以及方法的类。此节点将优化单个方法的超参数。
GeneticFeatureSelectorNode	使用进化算法优化一组特征，导出一个基本的sklearn选择器，该选择器简单地选择由节点选择的特征。
FSSNode	FSS代表FeatureSetSelector。该节点接收用户定义的特征子集列表，并选择一个预定义的子集。请注意，TPOT不会创建新的子集，也不会在每个节点中选择多个子集。如果使用线性管道，此节点应设置为第一步。在线性管道中，建议您仅使用少量特征集。我建议在允许TPOT一次选择多个FSSNode的管道中探索使用FSSNodes。例如，DynamicUnionPipeline和GraphPipeline都是FSSNode的优秀组合。在线性管道的开头使用DynamicUnionPipeline中的FFSNode，以探索线性管道中子集的最佳组合。设置GraphSearchPipeline的leaf_search_space，TPOT可以以不同的方式使用多个特征集，例如，为不同的集使用不同的转换器。

管道搜索空间¶

在 tpot2.search_spaces.pipelines 中找到

WrapperPipeline - 这个搜索空间用于包装一个sklearn估计器，该方法接受另一个估计器和超参数作为参数。例如，这可以与sklearn.ensemble.BaggingClassifier或sklearn.ensemble.AdaBoostClassifier一起使用。

名称	信息
ChoicePipeline	接收一个搜索空间列表。将从搜索空间中选择一个节点。
SequentialPipeline	接收一个搜索空间列表。将生成一个顺序长度的管道。管道中的每一步将对应于相同索引中提供的搜索空间。
DynamicLinearPipeline	接受一个单一的搜索空间。将生成一个可变长度的线性管道。管道中的每一步将从提供的搜索空间中提取。
UnionPipeline	接收一个搜索空间列表。返回的管道将包括每个搜索空间中的一个估计器，这些估计器在sklearn的FeatureUnion中连接。适用于在一个层中有许多步骤的情况。
DynamicUnionPipeline	接受一个单一的搜索空间。它将从搜索空间中提取1到max_estimators数量的估计器，并将它们在FeatureUnion中连接起来。
TreePipeline	生成一个可变长度的管道。管道将具有类似于TPOT1的树状结构。
GraphSearchPipeline	生成一个大小可变的有向无环图。如果需要，可以分别定义根节点、叶节点和内部节点的搜索空间。
WrapperPipeline	此搜索空间用于包装一个sklearn估计器，该方法接受另一个估计器和超参数作为参数。例如，这可以与sklearn.ensemble.BaggingClassifier或sklearn.ensemble.AdaBoostClassifier一起使用。

节点搜索空间示例¶

节点搜索空间代表sklearn管道的最小单元。所有节点搜索空间创建并优化一个单一节点，该节点导出一个单一的估计器对象。例如，这可以是一个KNeighborsClassifier或一个FeatureSetSelector。

EstimatorNode¶

EstimatorNode 表示 scikit-learn 估计器的超参数搜索空间。

In [3]:

Copied!





import tpot2
from ConfigSpace import ConfigurationSpace
from ConfigSpace import ConfigurationSpace, Integer, Float, Categorical, Normal
from sklearn.neighbors import KNeighborsClassifier

knn_configspace = ConfigurationSpace(
    space = {

        'n_neighbors': Integer("n_neighbors", bounds=(1, 10)),
        'weights': Categorical("weights", ['uniform', 'distance']),
        'p': Integer("p", bounds=(1, 3)),
        'metric': Categorical("metric", ['euclidean', 'minkowski']),
        'n_jobs': 1,
    }
)


knn_node = tpot2.search_spaces.nodes.EstimatorNode(
    method = KNeighborsClassifier,
    space = knn_configspace,
)
import tpot2
from ConfigSpace import ConfigurationSpace
from ConfigSpace import ConfigurationSpace, Integer, Float, Categorical, Normal
from sklearn.neighbors import KNeighborsClassifier

knn_configspace = ConfigurationSpace(
    space = {

        'n_neighbors': Integer("n_neighbors", bounds=(1, 10)),
        'weights': Categorical("weights", ['uniform', 'distance']),
        'p': Integer("p", bounds=(1, 3)),
        'metric': Categorical("metric", ['euclidean', 'minkowski']),
        'n_jobs': 1,
    }
)


knn_node = tpot2.search_spaces.nodes.EstimatorNode(
    method = KNeighborsClassifier,
    space = knn_configspace,
)

您可以使用generate()函数生成一个个体。这个个体从搜索空间中采样，并提供变异和交叉函数来修改当前样本。

In [4]:

Copied!

knn_individual = knn_node.generate()
knn_individual
knn_individual = knn_node.generate()
knn_individual

Out[4]:

<tpot2.search_spaces.nodes.estimator_node.EstimatorNodeIndividual at 0x78ec45f53430>

In [5]:

Copied!

print("sampled hyperparameters")
print(knn_individual.hyperparameters)
print("sampled hyperparameters")
print(knn_individual.hyperparameters)

sampled hyperparameters
{'metric': 'euclidean', 'n_jobs': 1, 'n_neighbors': 9, 'p': 1, 'weights': 'uniform'}

所有个体对象都有变异和交叉操作符，TPOT使用这些操作符来优化管道。

In [6]:

Copied!

knn_individual.mutate() # mutate the individual
print("mutated hyperparameters")
print(knn_individual.hyperparameters)
knn_individual.mutate() # mutate the individual
print("mutated hyperparameters")
print(knn_individual.hyperparameters)

mutated hyperparameters
{'metric': 'minkowski', 'n_jobs': 1, 'n_neighbors': 3, 'p': 3, 'weights': 'distance'}

在TPOT2中，交叉操作仅修改调用交叉函数的个体，第二个个体保持不变

In [7]:

Copied!





knn_individual1 = knn_node.generate()
knn_individual2 = knn_node.generate()

print("original hyperparameters for individual 1")
print(knn_individual1.hyperparameters)

print("original hyperparameters for individual 2")
print(knn_individual2.hyperparameters)

print()

knn_individual1.crossover(knn_individual2) # crossover the individuals
print("post crossover hyperparameters for individual 1")
print(knn_individual1.hyperparameters)
print("post crossover hyperparameters for individual 2")
print(knn_individual2.hyperparameters)
knn_individual1 = knn_node.generate()
knn_individual2 = knn_node.generate()

print("original hyperparameters for individual 1")
print(knn_individual1.hyperparameters)

print("original hyperparameters for individual 2")
print(knn_individual2.hyperparameters)

print()

knn_individual1.crossover(knn_individual2) # crossover the individuals
print("post crossover hyperparameters for individual 1")
print(knn_individual1.hyperparameters)
print("post crossover hyperparameters for individual 2")
print(knn_individual2.hyperparameters)

original hyperparameters for individual 1
{'metric': 'minkowski', 'n_jobs': 1, 'n_neighbors': 6, 'p': 2, 'weights': 'distance'}
original hyperparameters for individual 2
{'metric': 'euclidean', 'n_jobs': 1, 'n_neighbors': 4, 'p': 2, 'weights': 'uniform'}

post crossover hyperparameters for individual 1
{'metric': 'euclidean', 'n_jobs': 1, 'n_neighbors': 6, 'p': 2, 'weights': 'uniform'}
post crossover hyperparameters for individual 2
{'metric': 'euclidean', 'n_jobs': 1, 'n_neighbors': 4, 'p': 2, 'weights': 'uniform'}

所有搜索空间都有一个export_pipeline函数，该函数返回一个sklearn BaseEstimator

In [8]:

Copied!

est = knn_individual1.export_pipeline()
est
est = knn_individual1.export_pipeline()
est

Out[8]:

KNeighborsClassifier(metric='euclidean', n_jobs=1, n_neighbors=6)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

如果传递的是参数字典而不是ConfigSpace对象，则超参数将始终固定且不会被学习。

In [9]:

Copied!





import tpot2
from ConfigSpace import ConfigurationSpace
from ConfigSpace import ConfigurationSpace, Integer, Float, Categorical, Normal
from sklearn.neighbors import KNeighborsClassifier

space = {

    'n_neighbors':10,
}

knn_node = tpot2.search_spaces.nodes.EstimatorNode(
    method = KNeighborsClassifier,
    space = space,
)

knn_node.generate().export_pipeline()
import tpot2
from ConfigSpace import ConfigurationSpace
from ConfigSpace import ConfigurationSpace, Integer, Float, Categorical, Normal
from sklearn.neighbors import KNeighborsClassifier

space = {

    'n_neighbors':10,
}

knn_node = tpot2.search_spaces.nodes.EstimatorNode(
    method = KNeighborsClassifier,
    space = space,
)

knn_node.generate().export_pipeline()

Out[9]:

KNeighborsClassifier(n_neighbors=10)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

FSSNode 和 GeneticFeatureSelectorNode¶

这两个都有各自的教程。关于FFSNode的教程请参见教程3，关于GeneticFeatureSelectorNode的教程请参见教程5。

管道搜索空间示例¶

管道搜索空间用于定义TPOT可以搜索的管道的结构和限制。与节点搜索空间不同，所有管道搜索空间都将其他搜索空间作为输入。管道搜索空间可以从输入搜索空间中选择模型，并将它们组织在一个线性的sklearn Pipeline或TPOT GraphPipeline中，而不是采样超参数。

ChoicePipeline¶

最简单的管道搜索空间是ChoicePipeline。它接收一个搜索空间列表，并简单地从其中一个选择和采样。在这个例子中，我们将构建一个接收分类器多个选项的搜索空间。生成的搜索空间将首先从KNeighborsClassifier、LogisticRegression或DecisionTreeClassifier中选择一个模型，然后为给定的模型选择超参数。

In [10]:

Copied!





import tpot2
from ConfigSpace import ConfigurationSpace
from ConfigSpace import ConfigurationSpace, Integer, Float, Categorical, Normal
from sklearn.neighbors import KNeighborsClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier

knn_configspace = ConfigurationSpace(
    space = {

        'n_neighbors': Integer("n_neighbors", bounds=(1, 10)),
        'weights': Categorical("weights", ['uniform', 'distance']),
        'p': Integer("p", bounds=(1, 3)),
        'metric': Categorical("metric", ['euclidean', 'minkowski']),
        'n_jobs': 1,
    }
)

lr_configspace = ConfigurationSpace(
        space = {
            'solver': Categorical("solver", ['newton-cg', 'lbfgs', 'liblinear', 'sag', 'saga']),
            'penalty': Categorical("penalty", ['l1', 'l2']),
            'dual': Categorical("dual", [True, False]),
            'C': Float("C", bounds=(1e-4, 1e4), log=True),
            'class_weight': Categorical("class_weight", ['balanced']),
            'n_jobs': 1,
            'max_iter': 1000,
        }
    )

dt_configspace = ConfigurationSpace(
        space = {
            'criterion': Categorical("criterion", ['gini', 'entropy']),
            'max_depth': Integer("max_depth", bounds=(1, 11)),
            'min_samples_split': Integer("min_samples_split", bounds=(2, 21)),
            'min_samples_leaf': Integer("min_samples_leaf", bounds=(1, 21)),
            'max_features': Categorical("max_features", ['sqrt', 'log2']),
            'min_weight_fraction_leaf': 0.0,
        }
    )

knn_node = tpot2.search_spaces.nodes.EstimatorNode(
    method = KNeighborsClassifier,
    space = knn_configspace,
)

lr_node = tpot2.search_spaces.nodes.EstimatorNode(
    method = LogisticRegression,
    space = lr_configspace,
)

dt_node = tpot2.search_spaces.nodes.EstimatorNode(
    method = DecisionTreeClassifier,
    space = dt_configspace,
)

classifier_node = tpot2.search_spaces.pipelines.ChoicePipeline(
    search_spaces=[
        knn_node,
        lr_node,
        dt_node,
    ]
)


tpot2.search_spaces.pipelines.ChoicePipeline(
    search_spaces = [
        tpot2.search_spaces.nodes.EstimatorNode(
            method = KNeighborsClassifier,
            space = knn_configspace,
            ),
        tpot2.search_spaces.nodes.EstimatorNode(
            method = LogisticRegression,
            space = lr_configspace,
        ),
        tpot2.search_spaces.nodes.EstimatorNode(
            method = DecisionTreeClassifier,
            space = dt_configspace,
        ),
    ]
)
import tpot2
from ConfigSpace import ConfigurationSpace
from ConfigSpace import ConfigurationSpace, Integer, Float, Categorical, Normal
from sklearn.neighbors import KNeighborsClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier

knn_configspace = ConfigurationSpace(
    space = {

        'n_neighbors': Integer("n_neighbors", bounds=(1, 10)),
        'weights': Categorical("weights", ['uniform', 'distance']),
        'p': Integer("p", bounds=(1, 3)),
        'metric': Categorical("metric", ['euclidean', 'minkowski']),
        'n_jobs': 1,
    }
)

lr_configspace = ConfigurationSpace(
        space = {
            'solver': Categorical("solver", ['newton-cg', 'lbfgs', 'liblinear', 'sag', 'saga']),
            'penalty': Categorical("penalty", ['l1', 'l2']),
            'dual': Categorical("dual", [True, False]),
            'C': Float("C", bounds=(1e-4, 1e4), log=True),
            'class_weight': Categorical("class_weight", ['balanced']),
            'n_jobs': 1,
            'max_iter': 1000,
        }
    )

dt_configspace = ConfigurationSpace(
        space = {
            'criterion': Categorical("criterion", ['gini', 'entropy']),
            'max_depth': Integer("max_depth", bounds=(1, 11)),
            'min_samples_split': Integer("min_samples_split", bounds=(2, 21)),
            'min_samples_leaf': Integer("min_samples_leaf", bounds=(1, 21)),
            'max_features': Categorical("max_features", ['sqrt', 'log2']),
            'min_weight_fraction_leaf': 0.0,
        }
    )

knn_node = tpot2.search_spaces.nodes.EstimatorNode(
    method = KNeighborsClassifier,
    space = knn_configspace,
)

lr_node = tpot2.search_spaces.nodes.EstimatorNode(
    method = LogisticRegression,
    space = lr_configspace,
)

dt_node = tpot2.search_spaces.nodes.EstimatorNode(
    method = DecisionTreeClassifier,
    space = dt_configspace,
)

classifier_node = tpot2.search_spaces.pipelines.ChoicePipeline(
    search_spaces=[
        knn_node,
        lr_node,
        dt_node,
    ]
)


tpot2.search_spaces.pipelines.ChoicePipeline(
    search_spaces = [
        tpot2.search_spaces.nodes.EstimatorNode(
            method = KNeighborsClassifier,
            space = knn_configspace,
            ),
        tpot2.search_spaces.nodes.EstimatorNode(
            method = LogisticRegression,
            space = lr_configspace,
        ),
        tpot2.search_spaces.nodes.EstimatorNode(
            method = DecisionTreeClassifier,
            space = dt_configspace,
        ),
    ]
)

Out[10]:

<tpot2.search_spaces.pipelines.choice.ChoicePipeline at 0x78eb391763b0>

管道搜索空间提供的搜索空间对象与节点搜索空间的工作方式相同。请注意，交叉操作仅在两个个体都采样了相同方法时才有效。

In [11]:

Copied!

classifier_individual = classifier_node.generate()

print("sampled pipeline")
classifier_individual.export_pipeline()
classifier_individual = classifier_node.generate()

print("sampled pipeline")
classifier_individual.export_pipeline()

sampled pipeline

Out[11]:

LogisticRegression(C=0.0008500633703, class_weight='balanced', max_iter=1000,
                   n_jobs=1, penalty='l1', solver='saga')

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

In [12]:

Copied!

print("mutated pipeline")
classifier_individual.mutate()
classifier_individual.export_pipeline()
print("mutated pipeline")
classifier_individual.mutate()
classifier_individual.export_pipeline()

mutated pipeline

Out[12]:

LogisticRegression(C=0.1054489422979, class_weight='balanced', max_iter=1000,
                   n_jobs=1, penalty='l1', solver='liblinear')

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

EstimatorNode 和 ChoicePipeline 的内置搜索空间¶

TPOT2 还提供了预定义的超参数搜索空间。当前的搜索空间是从原始 TPOT 包以及 AutoSklearn 中使用的搜索空间组合中调整而来的。辅助函数 tpot2.config.get_search_space 接受一个字符串或字符串列表，并分别返回一个 EstimatorNode 或 ChoicePipeline（包括列表中的所有方法）。

字符串	对应方法
SGDClassifier
RandomForestClassifier
ExtraTreesClassifier
GradientBoostingClassifier
MLPClassifier
决策树分类器
XGBClassifier
KNeighborsClassifier
SVC
LogisticRegression
LGBMClassifier
LinearSVC
GaussianNB
BernoulliNB
MultinomialNB
ExtraTreesRegressor
RandomForestRegressor
GradientBoostingRegressor
BaggingRegressor
决策树回归器
KNeighborsRegressor
XGBRegressor
零计数
列独热编码器
二值化器
FastICA
特征聚合
最大绝对值缩放器
MinMaxScaler
标准化器
Nystroem
PCA
多项式特征
RBFSampler
RobustScaler
StandardScaler
SelectFwe
SelectPercentile
VarianceThreshold
SGDRegressor
岭回归
Lasso
弹性网络
拉尔斯
LassoLars
LassoLarsCV
RidgeCV
SVR
LinearSVR
AdaBoostRegressor
ElasticNetCV
AdaBoostClassifier
MLPRegressor
GaussianProcessRegressor
HistGradientBoostingClassifier
HistGradientBoostingRegressor
AddTransformer
mul_neg_1_Transformer
MulTransformer
SafeReciprocalTransformer
EQTransformer
NETransformer
GETransformer
GTTransformer
LETransformer
LTTransformer
MinTransformer
MaxTransformer
ZeroTransformer
OneTransformer
NTransformer
PowerTransformer
QuantileTransformer
ARDRegression
二次判别分析
PassiveAggressiveClassifier
线性判别分析
DominantEncoder
隐性编码器
杂种优势编码器
UnderDominanceEncoder
超显性编码器
高斯过程分类器
BaggingClassifier
LGBMRegressor
直通
SkipTransformer
PassKBinsDiscretizer
SimpleImputer
IterativeImputer
KNNImputer
MDR
连续MDR
ReliefF
SURF
SURFstar
MultiSURF
LinearRegression_sklearnex
Ridge_sklearnex
Lasso_sklearnex
ElasticNet_sklearnex
SVR_sklearnex
NuSVR_sklearnex
RandomForestRegressor_sklearnex
KNeighborsRegressor_sklearnex
RandomForestClassifier_sklearnex
KNeighborsClassifier_sklearnex
SVC_sklearnex
NuSVC_sklearnex
LogisticRegression_sklearnex

一些方法需要一个包装的估计器。为了同时考虑回归和分类，这些方法已根据其特殊的字符串进行了分组。

包装特殊字符串	备注
RFE_classification	使用学习的ExtraTreesClassifier进行FRE
RFE_regression	使用学习到的ExtraTreesRegressor进行RFE
SelectFromModel_classification	使用学习过的ExtraTreesClassifier进行SelectFromModel
SelectFromModel_regression	使用学习到的ExtraTreesRegressor进行SelectFromModel
IterativeImputer_learned_estimators	使用学习到的ExtraTreesRegressor的IterativeImputer

还有一些特殊的字符串，包含预定义的方法列表。这些将返回包含方法的ChoicePipeline。

特殊字符串列表	包含的方法
"选择器"	["SelectFwe", "SelectPercentile", "VarianceThreshold",]
"selectors_classification"	["SelectFwe", "SelectPercentile", "VarianceThreshold", "RFE_classification", "SelectFromModel_classification"]
"selectors_regression"	["SelectFwe", "SelectPercentile", "VarianceThreshold", "RFE_regression", "SelectFromModel_regression"]
"分类器"	["LGBMClassifier", "BaggingClassifier", 'AdaBoostClassifier', 'BernoulliNB', 'DecisionTreeClassifier', 'ExtraTreesClassifier', 'GaussianNB', 'HistGradientBoostingClassifier', 'KNeighborsClassifier','LinearDiscriminantAnalysis', 'LogisticRegression', "LinearSVC", "SVC", 'MLPClassifier', 'MultinomialNB', "QuadraticDiscriminantAnalysis", 'RandomForestClassifier', 'SGDClassifier', 'XGBClassifier']
"regressors"	["LGBMRegressor", 'AdaBoostRegressor', "ARDRegression", 'DecisionTreeRegressor', 'ExtraTreesRegressor', 'HistGradientBoostingRegressor', 'KNeighborsRegressor', 'LinearSVR', "MLPRegressor", 'RandomForestRegressor', 'SGDRegressor', 'SVR', 'XGBRegressor']
"transformers"	["PassKBinsDiscretizer", "Binarizer", "PCA", "ZeroCount", "ColumnOneHotEncoder", "FastICA", "FeatureAgglomeration", "Nystroem", "RBFSampler", "QuantileTransformer", "PowerTransformer"]
"scalers"	["MinMaxScaler", "RobustScaler", "StandardScaler", "MaxAbsScaler", "Normalizer", ]
"all_transformers"	["transformers", "scalers"]
"算术"	["AddTransformer", "mul_neg_1_Transformer", "MulTransformer", "SafeReciprocalTransformer", "EQTransformer", "NETransformer", "GETransformer", "GTTransformer", "LETransformer", "LTTransformer", "MinTransformer", "MaxTransformer"]
"imputers"	["SimpleImputer", "IterativeImputer", "KNNImputer"]
"skrebate"	["ReliefF", "SURF", "SURFstar", "MultiSURF"]
"genetic_encoders"	["DominantEncoder", "RecessiveEncoder", "HeterosisEncoder", "UnderDominanceEncoder", "OverDominanceEncoder"]
"classifiers_sklearnex"	["RandomForestClassifier_sklearnex", "LogisticRegression_sklearnex", "KNeighborsClassifier_sklearnex", "SVC_sklearnex","NuSVC_sklearnex"]
"regressors_sklearnex"	["LinearRegression_sklearnex", "Ridge_sklearnex", "Lasso_sklearnex", "ElasticNet_sklearnex", "SVR_sklearnex", "NuSVR_sklearnex", "RandomForestRegressor_sklearnex", "KNeighborsRegressor_sklearnex"]
"遗传编码器"	["DominantEncoder", "RecessiveEncoder", "HeterosisEncoder", "UnderDominanceEncoder", "OverDominanceEncoder"]

以下是一些使用get_search_space函数获取搜索空间的示例。

In [13]:

Copied!

#same pipeline search space as before.
classifier_choice = tpot2.config.get_search_space(["KNeighborsClassifier", "LogisticRegression", "DecisionTreeClassifier"])

print("sampled pipeline 1")
classifier_choice.generate().export_pipeline()
#same pipeline search space as before.
classifier_choice = tpot2.config.get_search_space(["KNeighborsClassifier", "LogisticRegression", "DecisionTreeClassifier"])

print("sampled pipeline 1")
classifier_choice.generate().export_pipeline()

sampled pipeline 1

Out[13]:

KNeighborsClassifier(n_jobs=1, n_neighbors=55, weights='distance')

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

In [14]:

Copied!

print("sampled pipeline 2")
classifier_choice.generate().export_pipeline()
print("sampled pipeline 2")
classifier_choice.generate().export_pipeline()

sampled pipeline 2

Out[14]:

LogisticRegression(C=0.012915602763, l1_ratio=0.2577823332886, max_iter=1000,
                   n_jobs=1, penalty='elasticnet', solver='saga')

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

In [15]:

Copied!

#search space for all classifiers
classifier_choice = tpot2.config.get_search_space("classifiers")

print("sampled pipeline 1")
classifier_choice.generate().export_pipeline()
#search space for all classifiers
classifier_choice = tpot2.config.get_search_space("classifiers")

print("sampled pipeline 1")
classifier_choice.generate().export_pipeline()

sampled pipeline 1

Out[15]:

SGDClassifier(alpha=0.0038384092036, class_weight='balanced',
              eta0=0.7197535254246, l1_ratio=0.8816063677431,
              loss='modified_huber', n_jobs=1, penalty='elasticnet')

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

In [16]:

Copied!

print("sampled pipeline 2")
classifier_choice.generate().export_pipeline()
print("sampled pipeline 2")
classifier_choice.generate().export_pipeline()

sampled pipeline 2

Out[16]:

KNeighborsClassifier(n_jobs=1, n_neighbors=1, p=1, weights='distance')

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

关于可重复性的说明¶

许多sklearn估计器，如RandomForestClassifier，是随机的，并且需要一个random_state参数以确保结果具有确定性。如果您希望TPOT运行可重现，重要的是TPOT使用的估计器设置了随机状态。TPOT不会自动设置此值。这可以在每个搜索空间中手动设置，或者通过将随机状态传递给get_search_space函数来设置。例如：

In [17]:

Copied!

reproducible_random_forest = tpot2.config.get_search_space("RandomForestClassifier", random_state=1)
reproducible_random_forest.generate().export_pipeline()
reproducible_random_forest = tpot2.config.get_search_space("RandomForestClassifier", random_state=1)
reproducible_random_forest.generate().export_pipeline()

Out[17]:

RandomForestClassifier(bootstrap=False, criterion='entropy',
                       max_features=0.0121463021153, min_samples_leaf=10,
                       min_samples_split=14, n_estimators=128, random_state=1)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

SequentialPipeline¶

SequentialPipelines 具有固定长度，并且每一步都从预定义的分布中进行采样。

In [18]:

Copied!





selector_choicepipeline = tpot2.config.get_search_space("VarianceThreshold")
transformer_choicepipeline =  tpot2.config.get_search_space("PCA")
classifier_choicepipeline = tpot2.config.get_search_space("LogisticRegression")

stc_pipeline = tpot2.search_spaces.pipelines.SequentialPipeline([
    selector_choicepipeline,
    transformer_choicepipeline,
    classifier_choicepipeline,
])

print("sampled pipeline")
stc_pipeline.generate().export_pipeline()
selector_choicepipeline = tpot2.config.get_search_space("VarianceThreshold")
transformer_choicepipeline =  tpot2.config.get_search_space("PCA")
classifier_choicepipeline = tpot2.config.get_search_space("LogisticRegression")

stc_pipeline = tpot2.search_spaces.pipelines.SequentialPipeline([
    selector_choicepipeline,
    transformer_choicepipeline,
    classifier_choicepipeline,
])

print("sampled pipeline")
stc_pipeline.generate().export_pipeline()

sampled pipeline

Out[18]:

Pipeline(steps=[('variancethreshold',
                 VarianceThreshold(threshold=0.0008293708451)),
                ('pca', PCA(n_components=0.5048643890372)),
                ('logisticregression',
                 LogisticRegression(C=7.7606337566295, class_weight='balanced',
                                    l1_ratio=0.123465163557, max_iter=1000,
                                    n_jobs=1, penalty='elasticnet',
                                    solver='saga'))])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

这是一个选择器-转换器-分类器表单的示例。

请注意，这次序列中的每一步都是一个ChoicePipeline。在这里，SequentialPipeline可以按顺序从提供的搜索空间中进行采样。

In [19]:

Copied!





selector_choicepipeline = tpot2.config.get_search_space("selectors")
transformer_choicepipeline =  tpot2.config.get_search_space("transformers")
classifier_choicepipeline = tpot2.config.get_search_space("classifiers")

stc_pipeline = tpot2.search_spaces.pipelines.SequentialPipeline([
    selector_choicepipeline,
    transformer_choicepipeline,
    classifier_choicepipeline,
])

print("sampled pipeline")
stc_pipeline.generate().export_pipeline()
selector_choicepipeline = tpot2.config.get_search_space("selectors")
transformer_choicepipeline =  tpot2.config.get_search_space("transformers")
classifier_choicepipeline = tpot2.config.get_search_space("classifiers")

stc_pipeline = tpot2.search_spaces.pipelines.SequentialPipeline([
    selector_choicepipeline,
    transformer_choicepipeline,
    classifier_choicepipeline,
])

print("sampled pipeline")
stc_pipeline.generate().export_pipeline()

sampled pipeline

Out[19]:

Pipeline(steps=[('variancethreshold',
                 VarianceThreshold(threshold=0.1215210592814)),
                ('fastica', FastICA(n_components=83)),
                ('baggingclassifier',
                 BaggingClassifier(bootstrap_features=True,
                                   max_features=0.9057563115025,
                                   max_samples=0.2313759070451, n_estimators=89,
                                   n_jobs=1))])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

In [20]:

Copied!

print("sampled pipeline")
stc_pipeline.generate().export_pipeline()
print("sampled pipeline")
stc_pipeline.generate().export_pipeline()

sampled pipeline

Out[20]:

Pipeline(steps=[('selectpercentile',
                 SelectPercentile(percentile=25.1697450346144)),
                ('kbinsdiscretizer',
                 KBinsDiscretizer(encode='onehot-dense', n_bins=40,
                                  strategy='uniform')),
                ('lineardiscriminantanalysis',
                 LinearDiscriminantAnalysis(shrinkage=0.755769834898,
                                            solver='eigen'))])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

动态线性管道¶

DynamicLinearPipeline 接受一个单一的搜索空间，并随机采样并将估计器放置在一个没有预定义序列的列表中。DynamicLinearPipeline 最常与 LinearPipeline 配对使用。常见的策略是使用 DynamicLinearPipeline 来优化一系列预处理或特征工程步骤，然后使用最终的分类器或回归器。

In [21]:

Copied!

import tpot2.config

linear_feature_engineering = tpot2.search_spaces.pipelines.DynamicLinearPipeline(search_space = tpot2.config.get_search_space(["all_transformers","selectors_classification"]), max_length=10)
print("sampled pipeline")
linear_feature_engineering.generate().export_pipeline()
import tpot2.config

linear_feature_engineering = tpot2.search_spaces.pipelines.DynamicLinearPipeline(search_space = tpot2.config.get_search_space(["all_transformers","selectors_classification"]), max_length=10)
print("sampled pipeline")
linear_feature_engineering.generate().export_pipeline()

sampled pipeline

Out[21]:

Pipeline(steps=[('rbfsampler',
                 RBFSampler(gamma=0.1991726671256, n_components=7)),
                ('zerocount', ZeroCount()),
                ('binarizer', Binarizer(threshold=0.5354245073766))])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

In [22]:

Copied!

print("sampled pipeline")
linear_feature_engineering.generate().export_pipeline()
print("sampled pipeline")
linear_feature_engineering.generate().export_pipeline()

sampled pipeline

Out[22]:

Pipeline(steps=[('selectfwe', SelectFwe(alpha=0.0014251225737)),
                ('powertransformer', PowerTransformer())])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

In [23]:

Copied!





full_search_space = tpot2.search_spaces.pipelines.SequentialPipeline([
    linear_feature_engineering,
    tpot2.config.get_search_space("classifiers"),
])

print("sampled pipeline")
full_search_space.generate().export_pipeline()
full_search_space = tpot2.search_spaces.pipelines.SequentialPipeline([
    linear_feature_engineering,
    tpot2.config.get_search_space("classifiers"),
])

print("sampled pipeline")
full_search_space.generate().export_pipeline()

sampled pipeline

Out[23]:

Pipeline(steps=[('pipeline',
                 Pipeline(steps=[('nystroem',
                                  Nystroem(gamma=0.3480554902065,
                                           kernel='sigmoid', n_components=20)),
                                 ('binarizer',
                                  Binarizer(threshold=0.6696149189758)),
                                 ('minmaxscaler', MinMaxScaler())])),
                ('multinomialnb', MultinomialNB(alpha=0.0016967794962))])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

In [24]:

Copied!

print("sampled pipeline")
full_search_space.generate().export_pipeline()
print("sampled pipeline")
full_search_space.generate().export_pipeline()

sampled pipeline

Out[24]:

Pipeline(steps=[('pipeline',
                 Pipeline(steps=[('zerocount', ZeroCount()),
                                 ('variancethreshold',
                                  VarianceThreshold(threshold=0.0020422211173)),
                                 ('binarizer',
                                  Binarizer(threshold=0.9681763702))])),
                ('bernoullinb',
                 BernoulliNB(alpha=0.0816524714629, fit_prior=False))])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

联合管道¶

当您希望在单个层中进行多个转换时，联合管道可能会很有用。另一种常见的策略是在进行转换的同时保留原始数据，这时可以使用转换器和直通进行联合。

In [25]:

Copied!





transform_and_passthrough = tpot2.search_spaces.pipelines.UnionPipeline([
    tpot2.config.get_search_space("transformers"),
    tpot2.config.get_search_space("Passthrough"),
])

transform_and_passthrough.generate().export_pipeline()
transform_and_passthrough = tpot2.search_spaces.pipelines.UnionPipeline([
    tpot2.config.get_search_space("transformers"),
    tpot2.config.get_search_space("Passthrough"),
])

transform_and_passthrough.generate().export_pipeline()

Out[25]:

FeatureUnion(transformer_list=[('fastica',
                                FastICA(algorithm='deflation',
                                        n_components=66)),
                               ('passthrough', Passthrough())])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

UnionPipelines 是扩展线性搜索空间能力的绝佳工具。

In [26]:

Copied!





stc_pipeline2 = tpot2.search_spaces.pipelines.SequentialPipeline([
    tpot2.config.get_search_space("selectors"),
    transform_and_passthrough,
    tpot2.config.get_search_space("classifiers"),
])

stc_pipeline2.generate().export_pipeline()
stc_pipeline2 = tpot2.search_spaces.pipelines.SequentialPipeline([
    tpot2.config.get_search_space("selectors"),
    transform_and_passthrough,
    tpot2.config.get_search_space("classifiers"),
])

stc_pipeline2.generate().export_pipeline()

Out[26]:

Pipeline(steps=[('variancethreshold',
                 VarianceThreshold(threshold=0.0009494718313)),
                ('featureunion',
                 FeatureUnion(transformer_list=[('binarizer',
                                                 Binarizer(threshold=0.8136655878085)),
                                                ('passthrough',
                                                 Passthrough())])),
                ('adaboostclassifier',
                 AdaBoostClassifier(learning_rate=0.1727096029044,
                                    n_estimators=446))])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

联合管道也可以用于创建“分支”，如果你试图创建一个树状的搜索空间。当与FeatureSetSelector节点（FSSNode）配对时，这尤其有用，因为每个分支可以为不同的特征子集学习不同的特征工程，例如。

In [27]:

Copied!





st_pipeline = tpot2.search_spaces.pipelines.SequentialPipeline([
    tpot2.config.get_search_space("selectors"),
    tpot2.config.get_search_space("transformers"),
])

branched_pipeline = tpot2.search_spaces.pipelines.SequentialPipeline([
    tpot2.search_spaces.pipelines.UnionPipeline([
        st_pipeline,
        st_pipeline,
    ]),
    tpot2.config.get_search_space("classifiers"),
])

branched_pipeline.generate().export_pipeline()
st_pipeline = tpot2.search_spaces.pipelines.SequentialPipeline([
    tpot2.config.get_search_space("selectors"),
    tpot2.config.get_search_space("transformers"),
])

branched_pipeline = tpot2.search_spaces.pipelines.SequentialPipeline([
    tpot2.search_spaces.pipelines.UnionPipeline([
        st_pipeline,
        st_pipeline,
    ]),
    tpot2.config.get_search_space("classifiers"),
])

branched_pipeline.generate().export_pipeline()

Out[27]:

Pipeline(steps=[('featureunion',
                 FeatureUnion(transformer_list=[('pipeline-1',
                                                 Pipeline(steps=[('variancethreshold',
                                                                  VarianceThreshold(threshold=0.1996640297479)),
                                                                 ('powertransformer',
                                                                  PowerTransformer())])),
                                                ('pipeline-2',
                                                 Pipeline(steps=[('selectfwe',
                                                                  SelectFwe(alpha=0.0045323854667)),
                                                                 ('fastica',
                                                                  FastICA(n_components=34))]))])),
                ('quadraticdiscriminantanalysis',
                 QuadraticDiscriminantAnalysis(reg_param=0.8833282196313))])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Pipeline?Documentation for PipelineiNot fitted

Pipeline(steps=[('featureunion',
                 FeatureUnion(transformer_list=[('pipeline-1',
                                                 Pipeline(steps=[('variancethreshold',
                                                                  VarianceThreshold(threshold=0.1996640297479)),
                                                                 ('powertransformer',
                                                                  PowerTransformer())])),
                                                ('pipeline-2',
                                                 Pipeline(steps=[('selectfwe',
                                                                  SelectFwe(alpha=0.0045323854667)),
                                                                 ('fastica',
                                                                  FastICA(n_components=34))]))])),
                ('quadraticdiscriminantanalysis',
                 QuadraticDiscriminantAnalysis(reg_param=0.8833282196313))])

featureunion: FeatureUnion?Documentation for featureunion: FeatureUnion

FeatureUnion(transformer_list=[('pipeline-1',
                                Pipeline(steps=[('variancethreshold',
                                                 VarianceThreshold(threshold=0.1996640297479)),
                                                ('powertransformer',
                                                 PowerTransformer())])),
                               ('pipeline-2',
                                Pipeline(steps=[('selectfwe',
                                                 SelectFwe(alpha=0.0045323854667)),
                                                ('fastica',
                                                 FastICA(n_components=34))]))])

pipeline-1

VarianceThreshold?Documentation for VarianceThreshold

VarianceThreshold(threshold=0.1996640297479)

PowerTransformer?Documentation for PowerTransformer

PowerTransformer()

pipeline-2

SelectFwe?Documentation for SelectFwe

SelectFwe(alpha=0.0045323854667)

FastICA?Documentation for FastICA

FastICA(n_components=34)

QuadraticDiscriminantAnalysis?Documentation for QuadraticDiscriminantAnalysis

QuadraticDiscriminantAnalysis(reg_param=0.8833282196313)

DynamicUnionPipeline¶

DynamicUnionPipeline 的工作方式与 UnionPipeline 类似。然而，UnionPipeline 是固定长度的，每个索引对应作为列表提供的搜索空间，而 DynamicUnionPipeline 则接受一个单一的搜索空间，并会采样一个或多个估计器/管道，并使用 FeatureUnion 将它们连接起来。

请注意，DynamicUnionPipeline 会检查管道的唯一性，因此它永远不会连接两个完全相同的管道。换句话说，特征联合中的所有步骤都将是唯一的。

当您想要多个转换器（或在某些情况下，管道）但不确定需要多少或哪些时，这可能很有用。

In [28]:

Copied!

dynamic_transformers = tpot2.search_spaces.pipelines.DynamicUnionPipeline(tpot2.config.get_search_space("transformers"), max_estimators=4)
dynamic_transformers.generate().export_pipeline()
dynamic_transformers = tpot2.search_spaces.pipelines.DynamicUnionPipeline(tpot2.config.get_search_space("transformers"), max_estimators=4)
dynamic_transformers.generate().export_pipeline()

Out[28]:

FeatureUnion(transformer_list=[('zerocount', ZeroCount()),
                               ('powertransformer', PowerTransformer())])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

一个好的策略可能是将其与特征联合中的Passthrough配对，以便输出所有转换以及原始数据。

In [29]:

Copied!





dynamic_transformers_with_passthrough = tpot2.search_spaces.pipelines.UnionPipeline([
    dynamic_transformers,
    tpot2.config.get_search_space("Passthrough")],
    )

dynamic_transformers_with_passthrough.generate().export_pipeline()
dynamic_transformers_with_passthrough = tpot2.search_spaces.pipelines.UnionPipeline([
    dynamic_transformers,
    tpot2.config.get_search_space("Passthrough")],
    )

dynamic_transformers_with_passthrough.generate().export_pipeline()

Out[29]:

FeatureUnion(transformer_list=[('featureunion',
                                FeatureUnion(transformer_list=[('powertransformer',
                                                                PowerTransformer())])),
                               ('passthrough', Passthrough())])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

In [30]:

Copied!





stc_pipeline3 = tpot2.search_spaces.pipelines.SequentialPipeline([
    tpot2.config.get_search_space("selectors"),
    dynamic_transformers_with_passthrough,
    tpot2.config.get_search_space("classifiers"),
])

stc_pipeline3.generate().export_pipeline()
stc_pipeline3 = tpot2.search_spaces.pipelines.SequentialPipeline([
    tpot2.config.get_search_space("selectors"),
    dynamic_transformers_with_passthrough,
    tpot2.config.get_search_space("classifiers"),
])

stc_pipeline3.generate().export_pipeline()

Out[30]:

Pipeline(steps=[('selectpercentile',
                 SelectPercentile(percentile=3.5688237635159)),
                ('featureunion',
                 FeatureUnion(transformer_list=[('featureunion',
                                                 FeatureUnion(transformer_list=[('featureagglomeration',
                                                                                 FeatureAgglomeration(n_clusters=28,
                                                                                                      pooling_func=<function max at 0x78ec455b4e30>))])),
                                                ('passthrough',
                                                 Passthrough())])),
                ('logisticregression',
                 LogisticRegression(C=9762.07332929782, max_iter=1000, n_jobs=1,
                                    solver='saga'))])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

WrapperPipeline¶

一些sklearn估计器将其他sklearn估计器作为参数。包装器管道用于同时调整原始估计器的超参数和内部估计器的超参数。实际上，WrapperPipeline中的内部估计器可以是使用本教程中描述的任何方法定义的任何搜索空间。

get_search_space 将自动为不需要内部估计器的 sklearn 估计器创建内部搜索空间。例如，"SelectFromModel_classification" 将返回以下搜索空间

In [31]:

Copied!





SelectFromModel_configspace_part = ConfigurationSpace(
                                        space = {
                                            'threshold': Float('threshold', bounds=(1e-4, 1.0), log=True),
                                        }
                                    )

extratrees_estimator_node = tpot2.config.get_search_space("ExtraTreesClassifier") #this exports an ExtraTreesClassifier node
extratrees_estimator_node.generate().export_pipeline()
SelectFromModel_configspace_part = ConfigurationSpace(
                                        space = {
                                            'threshold': Float('threshold', bounds=(1e-4, 1.0), log=True),
                                        }
                                    )

extratrees_estimator_node = tpot2.config.get_search_space("ExtraTreesClassifier") #this exports an ExtraTreesClassifier node
extratrees_estimator_node.generate().export_pipeline()

Out[31]:

ExtraTreesClassifier(class_weight='balanced', max_features=0.6642237575313,
                     min_samples_leaf=17, min_samples_split=3, n_jobs=1)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

In [32]:

Copied!





from sklearn.ensemble import ExtraTreesClassifier
from sklearn.feature_selection import SelectFromModel

select_from_model_wrapper_searchspace = tpot2.search_spaces.pipelines.WrapperPipeline(
            method=SelectFromModel,
            space = SelectFromModel_configspace_part,
            estimator_search_space= extratrees_estimator_node,
        )

select_from_model_wrapper_searchspace.generate().export_pipeline()
from sklearn.ensemble import ExtraTreesClassifier
from sklearn.feature_selection import SelectFromModel

select_from_model_wrapper_searchspace = tpot2.search_spaces.pipelines.WrapperPipeline(
            method=SelectFromModel,
            space = SelectFromModel_configspace_part,
            estimator_search_space= extratrees_estimator_node,
        )

select_from_model_wrapper_searchspace.generate().export_pipeline()

Out[32]:

SelectFromModel(estimator=ExtraTreesClassifier(bootstrap=True,
                                               class_weight='balanced',
                                               max_features=0.3007313724684,
                                               min_samples_leaf=12,
                                               min_samples_split=17, n_jobs=1),
                threshold=0.0048046738992)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

用于集成/内部分类器和回归器的WrapperPipeline策略（EstimatorTransformer）¶

Sklearn Pipelines 只允许分类器/回归器作为最后一步。所有其他步骤都需要实现一个 transform 函数。我们可以通过将其包装在另一个转换器类中来绕过这个限制，该转换器类在 transform() 函数中返回 predict 或 predict_proba 的输出。

要将分类器包装为转换器，您可以使用以下类：tpot2.builtin_modules.EstimatorTransformer。您可以使用method参数指定是否传递predict、predict_proba或decision function的输出。

cross_val_predict_cv¶

另一个考虑因素是是否使用cross_val_predict_cv。如果设置了此参数，在模型训练期间，任何不是最终预测器的分类器或回归器将使用sklearn.model_selection.cross_val_predict将样本外预测传递到模型的后续步骤中。模型仍将拟合完整数据，这些数据将在训练后用于预测。在样本外预测上训练下游模型通常可以防止过拟合并提高性能。原因是这为下游模型提供了一个估计，即上游模型在未见数据上的表现如何。否则，如果上游模型严重过拟合数据，下游模型可能只会学会盲目信任看似预测良好的模型，从而将过拟合传播到最终结果。

缺点是cross_val_predict_cv在计算上要求显著更高，对于给定的数据集可能不是必要的。

注意：对于GraphSearchPipeline来说，这不是必需的，因为导出的GraphPipeline估计器确实内置了对内部/回归器的支持。您可以在初始化GraphSearchPipeline对象时设置cross_val_predict_cv参数，而不是使用包装器。

In [33]:

Copied!

classifiers = tpot2.config.get_search_space("classifiers")
wrapped_estimators = tpot2.search_spaces.pipelines.WrapperPipeline(tpot2.builtin_modules.EstimatorTransformer, {}, classifiers)

est = wrapped_estimators.generate().export_pipeline() #returns an estimator with a transform function
est
classifiers = tpot2.config.get_search_space("classifiers")
wrapped_estimators = tpot2.search_spaces.pipelines.WrapperPipeline(tpot2.builtin_modules.EstimatorTransformer, {}, classifiers)

est = wrapped_estimators.generate().export_pipeline() #returns an estimator with a transform function
est

Out[33]:

EstimatorTransformer(estimator=SVC(C=140.9223338924506, gamma=0.0007253447995,
                                   max_iter=3000, probability=True,
                                   shrinking=False))

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

In [34]:

Copied!

import numpy as np
X, y = np.random.rand(100, 10), np.random.randint(0, 2, 100)

est.fit_transform(X, y)[0:5]
import numpy as np
X, y = np.random.rand(100, 10), np.random.randint(0, 2, 100)

est.fit_transform(X, y)[0:5]

Out[34]:

array([[0.5       , 0.5       ],
       [0.50964815, 0.49035185],
       [0.50681558, 0.49318442],
       [0.51565809, 0.48434191],
       [0.52006004, 0.47993996]])

你可以手动设置估计器的设置，就像你为EstimatorNode做的那样。这是另一个使用cross_val_predict和method的示例。

In [35]:

Copied!





classifiers = tpot2.config.get_search_space("classifiers")
wrapped_estimators_cv = tpot2.search_spaces.pipelines.WrapperPipeline(tpot2.builtin_modules.EstimatorTransformer, {'cross_val_predict_cv':10, 'method':'predict'}, classifiers)
est = wrapped_estimators_cv.generate().export_pipeline() #returns an estimator with a transform function
est.fit_transform(X, y)[0:5]
classifiers = tpot2.config.get_search_space("classifiers")
wrapped_estimators_cv = tpot2.search_spaces.pipelines.WrapperPipeline(tpot2.builtin_modules.EstimatorTransformer, {'cross_val_predict_cv':10, 'method':'predict'}, classifiers)
est = wrapped_estimators_cv.generate().export_pipeline() #returns an estimator with a transform function
est.fit_transform(X, y)[0:5]

Out[35]:

array([[0],
       [0],
       [1],
       [1],
       [1]])

这些现在可以在线性管道中使用。这与默认的线性管道搜索空间非常相似。

图搜索管道¶

GraphSearchPipeline 是一个灵活的搜索空间，没有对管道结构的先验限制。使用 GraphSearchPipeline，TPOT 将创建一个有向无环图形状的管道。在整个优化过程中，TPOT 可能会添加/删除节点，添加/删除边，并为每个节点执行模型选择和超参数调优。

graph_search_space 的主要参数是 root_search_space、inner_search_space 和 leaf_search_space。

参数	类型	描述
root_search_space	SklearnIndividualGenerator	图中根节点的搜索空间。该节点将是管道中的最终估计器。
inner_search_space	SklearnIndividualGenerator, optional	图中内部节点的搜索空间。如果未定义，则不会有内部节点。
leaf_search_space	SklearnIndividualGenerator, optional	图中叶子节点的搜索空间。如果未定义，叶子节点将从inner_search_space中抽取。
crossover_same_depth	bool, 可选	如果为True，交叉将仅在图中相同深度的节点之间发生。如果为False，交叉将在任何深度的节点之间发生。
cross_val_predict_cv	int, cross-validation generator or an iterable, optional	确定内部分类器或回归器中使用的交叉验证分割策略。
method	str, optional	用于内部分类器或回归器的预测方法。如果为'auto'，它将尝试按顺序使用predict_proba、decision_function或predict。

此搜索空间导出一个tpot2.GraphPipeline。这类似于scikit-learn的Pipeline，但用于有向无环图管道。您可以在教程6中了解更多关于使用此模块的信息。

In [37]:

Copied!





graph_search_space = tpot2.search_spaces.pipelines.GraphSearchPipeline(
    root_search_space= tpot2.config.get_search_space(["KNeighborsClassifier", "LogisticRegression", "DecisionTreeClassifier"]),
    leaf_search_space = tpot2.config.get_search_space("selectors"), 
    inner_search_space = tpot2.config.get_search_space(["transformers"]),
    max_size = 10,
)

ind = graph_search_space.generate()
graph_search_space = tpot2.search_spaces.pipelines.GraphSearchPipeline(
    root_search_space= tpot2.config.get_search_space(["KNeighborsClassifier", "LogisticRegression", "DecisionTreeClassifier"]),
    leaf_search_space = tpot2.config.get_search_space("selectors"), 
    inner_search_space = tpot2.config.get_search_space(["transformers"]),
    max_size = 10,
)

ind = graph_search_space.generate()

In [38]:

Copied!

est1 = ind.export_pipeline()
est1.plot() #GraphPipelines have a helpful plotting function to visualize the pipeline
est1 = ind.export_pipeline()
est1.plot() #GraphPipelines have a helpful plotting function to visualize the pipeline

No description has been provided for this image

让我们添加更多的变异并绘制最终的管道，以了解使用此搜索空间可以生成的管道的多样性

In [39]:

Copied!





for i in range(0,50):
    ind.mutate()
    if i%5==0:
        est = ind.export_pipeline()
        est.plot()
for i in range(0,50):
    ind.mutate()
    if i%5==0:
        est = ind.export_pipeline()
        est.plot()