TPOT2中的遗传特征选择节点¶
TPOT2 可以使用进化算法同时优化特征选择和管道优化。它包括两个具有不同特征选择策略的节点搜索空间：FSSNode 和 GeneticFeatureSelectorNode。
FSSNode - (特征集选择器) 如果你有一组预定义的特征集想要从中选择，这个节点非常有用。每个FeatureSetSelector节点将选择一组特征传递给管道中的下一步。请注意，FSSNode不会创建自己的特征子集，也不会混合/匹配多个预定义的特征集。
GeneticFeatureSelectorNode—与FSSNode从预定义的特征子集列表中选择不同，该节点使用进化算法从头开始优化一个新的特征子集。这在没有预定义特征分组的情况下非常有用。
本教程重点介绍FSSNode。有关GeneticFeatureSelectorNode的更多信息，请参见教程5。
将这些搜索空间与次要目标函数配对以最小化复杂性也可能是有益的。这将鼓励TPOT尝试生成具有最少特征的最简单管道。
tpot2.objectives.number_of_nodes_objective - 这可以用作另一个目标函数，用于计算节点的数量。
tpot2.objectives.complexity_scorer - 这是一个评分器，用于尝试计算学习参数的总数（系数的数量、决策树中的节点数等）。
特征集选择器¶
FeatureSetSelector 是 sklearn.feature_selection.SelectorMixin 的一个子类，它简单地返回手动指定的列。参数 sel_subset 指定了它选择的列的名称或索引。然后，transform 函数简单地索引并返回选定的列。您还可以选择使用 name 参数为组命名，尽管这只是为了记录，类本身并不使用它。
sel_subset: list or int
If X is a dataframe, items in sel_subset list must correspond to column names
If X is a numpy array, items in sel_subset list must correspond to column indexes
int: index of a single columnIn [1]:
Copied!
import tpot2
import pandas as pd
import numpy as np
#make a dataframe with columns a,b,c,d,e,f
#numpy array where columns are 1,2,3,4,5,6
data = np.repeat([np.arange(6)],10,0)
df = pd.DataFrame(data,columns=['a','b','c','d','e','f'])
fss = tpot2.builtin_modules.FeatureSetSelector(name='test',sel_subset=['a','b','c'])
print("original DataFrame")
print(df)
print("Transformed Data")
print(fss.fit_transform(df))
import tpot2
import pandas as pd
import numpy as np
#make a dataframe with columns a,b,c,d,e,f
#numpy array where columns are 1,2,3,4,5,6
data = np.repeat([np.arange(6)],10,0)
df = pd.DataFrame(data,columns=['a','b','c','d','e','f'])
fss = tpot2.builtin_modules.FeatureSetSelector(name='test',sel_subset=['a','b','c'])
print("original DataFrame")
print(df)
print("Transformed Data")
print(fss.fit_transform(df))
original DataFrame a b c d e f 0 0 1 2 3 4 5 1 0 1 2 3 4 5 2 0 1 2 3 4 5 3 0 1 2 3 4 5 4 0 1 2 3 4 5 5 0 1 2 3 4 5 6 0 1 2 3 4 5 7 0 1 2 3 4 5 8 0 1 2 3 4 5 9 0 1 2 3 4 5 Transformed Data [[0 1 2] [0 1 2] [0 1 2] [0 1 2] [0 1 2] [0 1 2] [0 1 2] [0 1 2] [0 1 2] [0 1 2]]
FSS节点¶
FSSNode 是一个节点搜索空间，它简单地从一组特征集中选择一个特征集。这与 EstimatorNode 的工作方式相同，但为定义特征集提供了更简单的接口。
请注意，FSS 仅在作为管道中的第一步使用时才有明确定义。这是因为下游节点将接收数据的不同转换，使得原始索引不再对应于转换后数据中的相同列。
FSSNode 接受一个参数 subsets，该参数定义了特征组。有四种定义子集的方式。
subsets : str or list, default=None
Sets the subsets that the FeatureSetSeletor will select from if set as an option in one of the configuration dictionaries.
Features are defined by column names if using a Pandas data frame, or ints corresponding to indexes if using numpy arrays.
- str : If a string, it is assumed to be a path to a csv file with the subsets.
The first column is assumed to be the name of the subset and the remaining columns are the features in the subset.
- list or np.ndarray : If a list or np.ndarray, it is assumed to be a list of subsets (i.e a list of lists).
- dict : A dictionary where keys are the names of the subsets and the values are the list of features.
- int : If an int, it is assumed to be the number of subsets to generate. Each subset will contain one feature.
- None : If None, each column will be treated as a subset. One column will be selected per subset.假设你想要有三组特征，每组各有三列。以下示例是等价的：
str¶
sel_subsets=simple_fss.csv
\# simple_fss.csv
group_one, 1,2,3
group_two, 4,5,6
group_three, 7,8,9字典¶
sel_subsets = { "group_one" : [1,2,3], "group_two" : [4,5,6], "group_three" : [7,8,9], }
列表¶
sel_subsets = [[1,2,3], [4,5,6], [7,8,9]]
示例¶
对于这些示例，我们创建了一个虚拟数据集，其中前六列是有信息的，其余的是无信息的。
In [2]:
Copied!
import tpot2
import sklearn.datasets
from sklearn.linear_model import LogisticRegression
import numpy as np
import pandas as pd
import tpot2
import sklearn.datasets
from sklearn.linear_model import LogisticRegression
import numpy as np
from tpot2.search_spaces.nodes import *
from tpot2.search_spaces.pipelines import *
from tpot2.config import get_search_space
X, y = sklearn.datasets.make_classification(n_samples=1000, n_features=6, n_informative=6, n_redundant=0, n_repeated=0, n_classes=2, n_clusters_per_class=2, weights=None, flip_y=0.01, class_sep=1.0, hypercube=True, shift=0.0, scale=1.0, shuffle=True, random_state=None)
X = np.hstack([X, np.random.rand(X.shape[0],6)]) #add six uninformative features
X = pd.DataFrame(X, columns=['a','b','c','d','e','f','g','h','i', 'j', 'k', 'l']) # a, b ,c the rest are uninformative
X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, train_size=0.75, test_size=0.25)
X.head()
import tpot2
import sklearn.datasets
from sklearn.linear_model import LogisticRegression
import numpy as np
import pandas as pd
import tpot2
import sklearn.datasets
from sklearn.linear_model import LogisticRegression
import numpy as np
from tpot2.search_spaces.nodes import *
from tpot2.search_spaces.pipelines import *
from tpot2.config import get_search_space
X, y = sklearn.datasets.make_classification(n_samples=1000, n_features=6, n_informative=6, n_redundant=0, n_repeated=0, n_classes=2, n_clusters_per_class=2, weights=None, flip_y=0.01, class_sep=1.0, hypercube=True, shift=0.0, scale=1.0, shuffle=True, random_state=None)
X = np.hstack([X, np.random.rand(X.shape[0],6)]) #add six uninformative features
X = pd.DataFrame(X, columns=['a','b','c','d','e','f','g','h','i', 'j', 'k', 'l']) # a, b ,c the rest are uninformative
X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, train_size=0.75, test_size=0.25)
X.head()
Out[2]:
| a | b | c | d | e | f | g | h | i | j | k | l | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | -0.988411 | -3.270714 | -1.816697 | 0.384124 | 1.258591 | -1.577232 | 0.101273 | 0.657975 | 0.770880 | 0.882366 | 0.637714 | 0.002812 |
| 1 | -0.531157 | -1.298541 | -2.630749 | 0.036662 | -2.097307 | -1.711751 | 0.894172 | 0.727579 | 0.211429 | 0.223319 | 0.496683 | 0.840040 |
| 2 | -0.896734 | -1.805453 | -2.736948 | -0.310169 | 1.802988 | -0.269441 | 0.765178 | 0.341713 | 0.847770 | 0.696190 | 0.824104 | 0.297523 |
| 3 | 1.637719 | -0.930537 | -0.229303 | 0.198907 | 1.184137 | -0.411545 | 0.870378 | 0.811312 | 0.142528 | 0.707361 | 0.201967 | 0.867956 |
| 4 | -1.709777 | -2.701615 | 0.297434 | -0.909832 | 1.436884 | 0.120985 | 0.866854 | 0.352461 | 0.690270 | 0.172950 | 0.056518 | 0.806867 |
假设基于先前的知识或兴趣，我们知道特征可以如下分组
In [3]:
Copied!
subsets = { "group_one" : ['a','b','c',],
"group_two" : ['d','e','f'],
"group_three" : ['g','h','i'],
"group_four" : ['j','k','l'],
}
subsets = { "group_one" : ['a','b','c',],
"group_two" : ['d','e','f'],
"group_three" : ['g','h','i'],
"group_four" : ['j','k','l'],
}
我们可以创建一个FSSNode，它将从这个子集中进行选择。管道中的每个节点只选择一个子集。
In [4]:
Copied!
fss_search_space = FSSNode(subsets=subsets)
fss_search_space = FSSNode(subsets=subsets)
如果我们从这个搜索空间中随机抽样，我们可以看到我们得到了一个选择器，它选择了预定义集合中的一个。在这种情况下，它选择了第二组，其中包括['d', 'e', 'f']。（在生成函数中设置了一个随机种子，以便在重新运行笔记本时选择相同的组。）
In [5]:
Copied!
fss_selector = fss_search_space.generate(rng=1).export_pipeline()
fss_selector
fss_selector = fss_search_space.generate(rng=1).export_pipeline()
fss_selector
Out[5]:
FeatureSetSelector(name='group_two', sel_subset=['d', 'e', 'f'])In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
FeatureSetSelector(name='group_two', sel_subset=['d', 'e', 'f'])
In [6]:
Copied!
fss_selector.set_output(transform="pandas") #by default sklearn selectors return numpy arrays. this will make it return pandas dataframes
fss_selector.fit(X_train)
fss_selector.transform(X_train)
fss_selector.set_output(transform="pandas") #by default sklearn selectors return numpy arrays. this will make it return pandas dataframes
fss_selector.fit(X_train)
fss_selector.transform(X_train)
Out[6]:
| d | e | f | |
|---|---|---|---|
| 28 | -2.393671 | 2.653494 | 1.336840 |
| 540 | -1.598037 | -2.639941 | -1.787062 |
| 980 | -1.562249 | 1.573867 | -0.135207 |
| 812 | 0.084835 | 1.809188 | -1.525609 |
| 117 | 0.647414 | 1.437139 | 1.873279 |
| ... | ... | ... | ... |
| 630 | 0.102721 | 0.463829 | -0.220689 |
| 963 | -0.530709 | 0.353686 | 0.621369 |
| 943 | 3.850193 | 0.948248 | -2.042764 |
| 930 | 1.051634 | 1.240570 | -1.477092 |
| 116 | -0.126476 | -1.599799 | -0.610169 |
750 行 × 3 列
在底层，突变将随机选择另一个特征集，而交叉将交换由两个个体选择的特征集
In [7]:
Copied!
ind1 = fss_search_space.generate(rng=1)
ind1.export_pipeline()
ind1 = fss_search_space.generate(rng=1)
ind1.export_pipeline()
Out[7]:
FeatureSetSelector(name='group_two', sel_subset=['d', 'e', 'f'])In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
FeatureSetSelector(name='group_two', sel_subset=['d', 'e', 'f'])
In [8]:
Copied!
ind1.mutate()
ind1.export_pipeline()
ind1.mutate()
ind1.export_pipeline()
Out[8]:
FeatureSetSelector(name='group_three', sel_subset=['g', 'h', 'i'])In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
FeatureSetSelector(name='group_three', sel_subset=['g', 'h', 'i'])
我们现在可以在定义我们的管道时使用这个。 对于第一个例子，我们将构建一个简单的线性管道，其中第一步是特征集选择器，第二步是分类器
In [9]:
Copied!
classification_search_space = get_search_space(["RandomForestClassifier"])
fss_and_classifier_search_space = SequentialPipeline([fss_search_space, classification_search_space])
est = tpot2.TPOTEstimator(generations=5,
scorers=["roc_auc_ovr", tpot2.objectives.complexity_scorer],
scorers_weights=[1.0, -1.0],
n_jobs=32,
classification=True,
search_space = fss_and_classifier_search_space,
verbose=1,
)
scorer = sklearn.metrics.get_scorer('roc_auc_ovr')
est.fit(X_train, y_train)
print(scorer(est, X_test, y_test))
classification_search_space = get_search_space(["RandomForestClassifier"])
fss_and_classifier_search_space = SequentialPipeline([fss_search_space, classification_search_space])
est = tpot2.TPOTEstimator(generations=5,
scorers=["roc_auc_ovr", tpot2.objectives.complexity_scorer],
scorers_weights=[1.0, -1.0],
n_jobs=32,
classification=True,
search_space = fss_and_classifier_search_space,
verbose=1,
)
scorer = sklearn.metrics.get_scorer('roc_auc_ovr')
est.fit(X_train, y_train)
print(scorer(est, X_test, y_test))
Generation: 100%|██████████| 5/5 [00:30<00:00, 6.11s/it]
0.90263107355483
In [10]:
Copied!
est.fitted_pipeline_
est.fitted_pipeline_
Out[10]:
Pipeline(steps=[('featuresetselector',
FeatureSetSelector(name='group_one',
sel_subset=['a', 'b', 'c'])),
('randomforestclassifier',
RandomForestClassifier(criterion='entropy',
max_features=0.4070021568844,
min_samples_leaf=4, min_samples_split=3,
n_estimators=128))])In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
Pipeline(steps=[('featuresetselector',
FeatureSetSelector(name='group_one',
sel_subset=['a', 'b', 'c'])),
('randomforestclassifier',
RandomForestClassifier(criterion='entropy',
max_features=0.4070021568844,
min_samples_leaf=4, min_samples_split=3,
n_estimators=128))])FeatureSetSelector(name='group_one', sel_subset=['a', 'b', 'c'])
RandomForestClassifier(criterion='entropy', max_features=0.4070021568844,
min_samples_leaf=4, min_samples_split=3,
n_estimators=128)通过这种设置，TPOT能够识别出使用的一个子集，但性能并不理想。在这种情况下，我们恰好知道需要多个特征集。如果我们想在管道中包含多个特征，我们将不得不修改我们的搜索空间。对此有三种选择。
- UnionPipeline - 这允许您选择固定数量的特征集。如果您使用带有两个FSSNodes的UnionPipeline，您将始终选择两个特征集，它们只是简单地连接在一起。
- DynamicUnionPipeline - 此空间允许选择多个FSSNodes。与UnionPipeline不同，您不必指定所选集合的数量，TPOT将识别最优的集合数量。此外，使用DynamicUnionPipeline时，相同的特征集不能被选择两次。请注意，虽然DynamicUnionPipeline可以选择多个特征集，但它永远不会将两个特征集混合在一起。
- GraphSearchPipeline - 当设置为leave_search_space时，GraphSearchPipeline还可以选择多个FSSNodes，这些FSSNodes作为管道的其余部分的输入。
UnionPipeline + FSSNode 示例¶
In [11]:
Copied!
union_fss_space = UnionPipeline([fss_search_space, fss_search_space])
union_fss_space = UnionPipeline([fss_search_space, fss_search_space])
In [12]:
Copied!
# this union search space will always select exactly two fss_search_space
selector1 = union_fss_space.generate(rng=1).export_pipeline()
selector1
# this union search space will always select exactly two fss_search_space
selector1 = union_fss_space.generate(rng=1).export_pipeline()
selector1
Out[12]:
FeatureUnion(transformer_list=[('featuresetselector-1',
FeatureSetSelector(name='group_two',
sel_subset=['d', 'e', 'f'])),
('featuresetselector-2',
FeatureSetSelector(name='group_three',
sel_subset=['g', 'h',
'i']))])In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
FeatureUnion(transformer_list=[('featuresetselector-1',
FeatureSetSelector(name='group_two',
sel_subset=['d', 'e', 'f'])),
('featuresetselector-2',
FeatureSetSelector(name='group_three',
sel_subset=['g', 'h',
'i']))])FeatureSetSelector(name='group_two', sel_subset=['d', 'e', 'f'])
FeatureSetSelector(name='group_three', sel_subset=['g', 'h', 'i'])
In [13]:
Copied!
selector1.set_output(transform="pandas")
selector1.fit(X_train)
selector1.transform(X_train)
selector1.set_output(transform="pandas")
selector1.fit(X_train)
selector1.transform(X_train)
Out[13]:
| d | e | f | g | h | i | |
|---|---|---|---|---|---|---|
| 28 | -2.393671 | 2.653494 | 1.336840 | 0.671229 | 0.431712 | 0.090788 |
| 540 | -1.598037 | -2.639941 | -1.787062 | 0.520648 | 0.436337 | 0.576560 |
| 980 | -1.562249 | 1.573867 | -0.135207 | 0.323676 | 0.052558 | 0.892457 |
| 812 | 0.084835 | 1.809188 | -1.525609 | 0.777859 | 0.327459 | 0.626609 |
| 117 | 0.647414 | 1.437139 | 1.873279 | 0.383676 | 0.448043 | 0.908426 |
| ... | ... | ... | ... | ... | ... | ... |
| 630 | 0.102721 | 0.463829 | -0.220689 | 0.155922 | 0.057284 | 0.581789 |
| 963 | -0.530709 | 0.353686 | 0.621369 | 0.701410 | 0.205080 | 0.189494 |
| 943 | 3.850193 | 0.948248 | -2.042764 | 0.737312 | 0.082513 | 0.886070 |
| 930 | 1.051634 | 1.240570 | -1.477092 | 0.207093 | 0.349121 | 0.027916 |
| 116 | -0.126476 | -1.599799 | -0.610169 | 0.185323 | 0.024521 | 0.685559 |
750 行 × 6 列
DynamicUnionPipeline + FSSNode 示例¶
动态联合管道可以选择可变数量的特征集。
In [14]:
Copied!
dynamic_fss_space = DynamicUnionPipeline(fss_search_space)
dynamic_fss_space.generate(rng=1).export_pipeline()
dynamic_fss_space = DynamicUnionPipeline(fss_search_space)
dynamic_fss_space.generate(rng=1).export_pipeline()
Out[14]:
FeatureUnion(transformer_list=[('featuresetselector',
FeatureSetSelector(name='group_three',
sel_subset=['g', 'h',
'i']))])In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
FeatureUnion(transformer_list=[('featuresetselector',
FeatureSetSelector(name='group_three',
sel_subset=['g', 'h',
'i']))])FeatureSetSelector(name='group_three', sel_subset=['g', 'h', 'i'])
In [15]:
Copied!
dynamic_fss_space.generate(rng=3).export_pipeline()
dynamic_fss_space.generate(rng=3).export_pipeline()
Out[15]:
FeatureUnion(transformer_list=[('featuresetselector-1',
FeatureSetSelector(name='group_one',
sel_subset=['a', 'b', 'c'])),
('featuresetselector-2',
FeatureSetSelector(name='group_four',
sel_subset=['j', 'k',
'l']))])In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
FeatureUnion(transformer_list=[('featuresetselector-1',
FeatureSetSelector(name='group_one',
sel_subset=['a', 'b', 'c'])),
('featuresetselector-2',
FeatureSetSelector(name='group_four',
sel_subset=['j', 'k',
'l']))])FeatureSetSelector(name='group_one', sel_subset=['a', 'b', 'c'])
FeatureSetSelector(name='group_four', sel_subset=['j', 'k', 'l'])
In [16]:
Copied!
graph_search_space = tpot2.search_spaces.pipelines.GraphSearchPipeline(
leaf_search_space = fss_search_space,
inner_search_space = tpot2.config.get_search_space(["transformers"]),
root_search_space= tpot2.config.get_search_space(["KNeighborsClassifier", "LogisticRegression", "DecisionTreeClassifier"]),
max_size = 10,
)
graph_search_space.generate(rng=4).export_pipeline().plot()
graph_search_space = tpot2.search_spaces.pipelines.GraphSearchPipeline(
leaf_search_space = fss_search_space,
inner_search_space = tpot2.config.get_search_space(["transformers"]),
root_search_space= tpot2.config.get_search_space(["KNeighborsClassifier", "LogisticRegression", "DecisionTreeClassifier"]),
max_size = 10,
)
graph_search_space.generate(rng=4).export_pipeline().plot()
使用TPOT进行优化¶
对于这个例子，我们将优化DynamicUnion搜索空间
In [17]:
Copied!
import tpot2
import sklearn.datasets
from sklearn.linear_model import LogisticRegression
import numpy as np
final_classification_search_space = SequentialPipeline([dynamic_fss_space, classification_search_space])
est = tpot2.TPOTEstimator(generations=5,
scorers=["roc_auc_ovr", tpot2.objectives.complexity_scorer],
scorers_weights=[1.0, -1.0],
n_jobs=32,
classification=True,
search_space = final_classification_search_space,
verbose=1,
)
scorer = sklearn.metrics.get_scorer('roc_auc_ovr')
est.fit(X_train, y_train)
print(scorer(est, X_test, y_test))
import tpot2
import sklearn.datasets
from sklearn.linear_model import LogisticRegression
import numpy as np
final_classification_search_space = SequentialPipeline([dynamic_fss_space, classification_search_space])
est = tpot2.TPOTEstimator(generations=5,
scorers=["roc_auc_ovr", tpot2.objectives.complexity_scorer],
scorers_weights=[1.0, -1.0],
n_jobs=32,
classification=True,
search_space = final_classification_search_space,
verbose=1,
)
scorer = sklearn.metrics.get_scorer('roc_auc_ovr')
est.fit(X_train, y_train)
print(scorer(est, X_test, y_test))
Generation: 100%|██████████| 5/5 [00:34<00:00, 6.88s/it]
0.9482747583381345
我们可以看到，这个管道表现得稍微好一些，并正确地将第一组和第二组识别为生成方程中使用的特征集。
In [18]:
Copied!
est.fitted_pipeline_
est.fitted_pipeline_
Out[18]:
Pipeline(steps=[('featureunion',
FeatureUnion(transformer_list=[('featuresetselector-1',
FeatureSetSelector(name='group_one',
sel_subset=['a',
'b',
'c'])),
('featuresetselector-2',
FeatureSetSelector(name='group_two',
sel_subset=['d',
'e',
'f']))])),
('randomforestclassifier',
RandomForestClassifier(bootstrap=False,
class_weight='balanced',
max_features=0.4909664847192,
min_samples_leaf=2, min_samples_split=4,
n_estimators=128))])In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
Pipeline(steps=[('featureunion',
FeatureUnion(transformer_list=[('featuresetselector-1',
FeatureSetSelector(name='group_one',
sel_subset=['a',
'b',
'c'])),
('featuresetselector-2',
FeatureSetSelector(name='group_two',
sel_subset=['d',
'e',
'f']))])),
('randomforestclassifier',
RandomForestClassifier(bootstrap=False,
class_weight='balanced',
max_features=0.4909664847192,
min_samples_leaf=2, min_samples_split=4,
n_estimators=128))])FeatureUnion(transformer_list=[('featuresetselector-1',
FeatureSetSelector(name='group_one',
sel_subset=['a', 'b', 'c'])),
('featuresetselector-2',
FeatureSetSelector(name='group_two',
sel_subset=['d', 'e',
'f']))])FeatureSetSelector(name='group_one', sel_subset=['a', 'b', 'c'])
FeatureSetSelector(name='group_two', sel_subset=['d', 'e', 'f'])
RandomForestClassifier(bootstrap=False, class_weight='balanced',
max_features=0.4909664847192, min_samples_leaf=2,
min_samples_split=4, n_estimators=128)In [27]:
Copied!
linear_search_space = tpot2.config.template_search_spaces.get_template_search_spaces("linear", classification=True)
fss_and_linear_search_space = SequentialPipeline([fss_search_space, linear_search_space])
# est = tpot2.TPOTEstimator(
# population_size=32,
# generations=10,
# scorers=["roc_auc_ovr", tpot2.objectives.complexity_scorer],
# scorers_weights=[1.0, -1.0],
# other_objective_functions=[number_of_selected_features],
# other_objective_functions_weights = [-1],
# objective_function_names = ["Number of selected features"],
# n_jobs=32,
# classification=True,
# search_space = fss_and_linear_search_space,
# verbose=2,
# )
fss_and_linear_search_space.generate(rng=1).export_pipeline()
linear_search_space = tpot2.config.template_search_spaces.get_template_search_spaces("linear", classification=True)
fss_and_linear_search_space = SequentialPipeline([fss_search_space, linear_search_space])
# est = tpot2.TPOTEstimator(
# population_size=32,
# generations=10,
# scorers=["roc_auc_ovr", tpot2.objectives.complexity_scorer],
# scorers_weights=[1.0, -1.0],
# other_objective_functions=[number_of_selected_features],
# other_objective_functions_weights = [-1],
# objective_function_names = ["Number of selected features"],
# n_jobs=32,
# classification=True,
# search_space = fss_and_linear_search_space,
# verbose=2,
# )
fss_and_linear_search_space.generate(rng=1).export_pipeline()
Out[27]:
Pipeline(steps=[('featuresetselector',
FeatureSetSelector(name='group_two', sel_subset=[3, 4, 5])),
('pipeline',
Pipeline(steps=[('maxabsscaler', MaxAbsScaler()),
('rfe',
RFE(estimator=ExtraTreesClassifier(max_features=0.0390676831531,
min_samples_leaf=8,
min_samples_split=14,
n_jobs=1),
step=0.753983388654)),
('featureunion-1',
FeatureUnion(transformer_list=[('f...
FeatureUnion(transformer_list=[('powertransformer',
PowerTransformer()),
('pca',
PCA(n_components=0.9286371732844))])),
('passthrough',
Passthrough())])),
('featureunion-2',
FeatureUnion(transformer_list=[('skiptransformer',
SkipTransformer()),
('passthrough',
Passthrough())])),
('kneighborsclassifier',
KNeighborsClassifier(n_jobs=1, n_neighbors=21,
weights='distance'))]))])In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
Pipeline(steps=[('featuresetselector',
FeatureSetSelector(name='group_two', sel_subset=[3, 4, 5])),
('pipeline',
Pipeline(steps=[('maxabsscaler', MaxAbsScaler()),
('rfe',
RFE(estimator=ExtraTreesClassifier(max_features=0.0390676831531,
min_samples_leaf=8,
min_samples_split=14,
n_jobs=1),
step=0.753983388654)),
('featureunion-1',
FeatureUnion(transformer_list=[('f...
FeatureUnion(transformer_list=[('powertransformer',
PowerTransformer()),
('pca',
PCA(n_components=0.9286371732844))])),
('passthrough',
Passthrough())])),
('featureunion-2',
FeatureUnion(transformer_list=[('skiptransformer',
SkipTransformer()),
('passthrough',
Passthrough())])),
('kneighborsclassifier',
KNeighborsClassifier(n_jobs=1, n_neighbors=21,
weights='distance'))]))])FeatureSetSelector(name='group_two', sel_subset=[3, 4, 5])
Pipeline(steps=[('maxabsscaler', MaxAbsScaler()),
('rfe',
RFE(estimator=ExtraTreesClassifier(max_features=0.0390676831531,
min_samples_leaf=8,
min_samples_split=14,
n_jobs=1),
step=0.753983388654)),
('featureunion-1',
FeatureUnion(transformer_list=[('featureunion',
FeatureUnion(transformer_list=[('powertransformer',
PowerTransformer()),
('pca',
PCA(n_components=0.9286371732844))])),
('passthrough',
Passthrough())])),
('featureunion-2',
FeatureUnion(transformer_list=[('skiptransformer',
SkipTransformer()),
('passthrough',
Passthrough())])),
('kneighborsclassifier',
KNeighborsClassifier(n_jobs=1, n_neighbors=21,
weights='distance'))])MaxAbsScaler()
RFE(estimator=ExtraTreesClassifier(max_features=0.0390676831531,
min_samples_leaf=8, min_samples_split=14,
n_jobs=1),
step=0.753983388654)ExtraTreesClassifier(max_features=0.0390676831531, min_samples_leaf=8,
min_samples_split=14, n_jobs=1)ExtraTreesClassifier(max_features=0.0390676831531, min_samples_leaf=8,
min_samples_split=14, n_jobs=1)FeatureUnion(transformer_list=[('featureunion',
FeatureUnion(transformer_list=[('powertransformer',
PowerTransformer()),
('pca',
PCA(n_components=0.9286371732844))])),
('passthrough', Passthrough())])PowerTransformer()
PCA(n_components=0.9286371732844)
Passthrough()
FeatureUnion(transformer_list=[('skiptransformer', SkipTransformer()),
('passthrough', Passthrough())])SkipTransformer()
Passthrough()
KNeighborsClassifier(n_jobs=1, n_neighbors=21, weights='distance')
变得高级¶
如果你想更高级一点，你可以结合更多的搜索空间，以便为每个特征集设置独特的预处理管道。以下是一个示例：
In [19]:
Copied!
dynamic_transformers = DynamicUnionPipeline(get_search_space("all_transformers"), max_estimators=4)
dynamic_transformers_with_passthrough = tpot2.search_spaces.pipelines.UnionPipeline([
dynamic_transformers,
tpot2.config.get_search_space("Passthrough")],
)
multi_step_engineering = DynamicLinearPipeline(dynamic_transformers_with_passthrough, max_length=4)
fss_engineering_search_space = SequentialPipeline([fss_search_space, multi_step_engineering])
union_fss_engineering_search_space = DynamicUnionPipeline(fss_engineering_search_space)
final_fancy_search_space = SequentialPipeline([union_fss_engineering_search_space, classification_search_space])
dynamic_transformers = DynamicUnionPipeline(get_search_space("all_transformers"), max_estimators=4)
dynamic_transformers_with_passthrough = tpot2.search_spaces.pipelines.UnionPipeline([
dynamic_transformers,
tpot2.config.get_search_space("Passthrough")],
)
multi_step_engineering = DynamicLinearPipeline(dynamic_transformers_with_passthrough, max_length=4)
fss_engineering_search_space = SequentialPipeline([fss_search_space, multi_step_engineering])
union_fss_engineering_search_space = DynamicUnionPipeline(fss_engineering_search_space)
final_fancy_search_space = SequentialPipeline([union_fss_engineering_search_space, classification_search_space])
In [20]:
Copied!
final_fancy_search_space.generate(rng=3).export_pipeline()
final_fancy_search_space.generate(rng=3).export_pipeline()
Out[20]:
Pipeline(steps=[('featureunion',
FeatureUnion(transformer_list=[('pipeline-1',
Pipeline(steps=[('featuresetselector',
FeatureSetSelector(name='group_one',
sel_subset=['a',
'b',
'c'])),
('pipeline',
Pipeline(steps=[('featureunion',
FeatureUnion(transformer_list=[('featureunion',
FeatureUnion(transformer_list=[('pca',
PCA(n_components=0.93113403057))])),
('passt...
FeatureUnion(transformer_list=[('featureunion',
FeatureUnion(transformer_list=[('quantiletransformer',
QuantileTransformer(n_quantiles=87)),
('columnonehotencoder',
ColumnOneHotEncoder())])),
('passthrough',
Passthrough())]))]))]))])),
('randomforestclassifier',
RandomForestClassifier(class_weight='balanced',
criterion='entropy',
max_features=0.021545996678,
min_samples_leaf=11,
n_estimators=128))])In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
Pipeline(steps=[('featureunion',
FeatureUnion(transformer_list=[('pipeline-1',
Pipeline(steps=[('featuresetselector',
FeatureSetSelector(name='group_one',
sel_subset=['a',
'b',
'c'])),
('pipeline',
Pipeline(steps=[('featureunion',
FeatureUnion(transformer_list=[('featureunion',
FeatureUnion(transformer_list=[('pca',
PCA(n_components=0.93113403057))])),
('passt...
FeatureUnion(transformer_list=[('featureunion',
FeatureUnion(transformer_list=[('quantiletransformer',
QuantileTransformer(n_quantiles=87)),
('columnonehotencoder',
ColumnOneHotEncoder())])),
('passthrough',
Passthrough())]))]))]))])),
('randomforestclassifier',
RandomForestClassifier(class_weight='balanced',
criterion='entropy',
max_features=0.021545996678,
min_samples_leaf=11,
n_estimators=128))])FeatureUnion(transformer_list=[('pipeline-1',
Pipeline(steps=[('featuresetselector',
FeatureSetSelector(name='group_one',
sel_subset=['a',
'b',
'c'])),
('pipeline',
Pipeline(steps=[('featureunion',
FeatureUnion(transformer_list=[('featureunion',
FeatureUnion(transformer_list=[('pca',
PCA(n_components=0.93113403057))])),
('passthrough',
Passthrough())]))]))])),...
FeatureUnion(transformer_list=[('binarizer',
Binarizer(threshold=0.5396272782675))])),
('passthrough',
Passthrough())])),
('featureunion-2',
FeatureUnion(transformer_list=[('featureunion',
FeatureUnion(transformer_list=[('quantiletransformer',
QuantileTransformer(n_quantiles=87)),
('columnonehotencoder',
ColumnOneHotEncoder())])),
('passthrough',
Passthrough())]))]))]))])FeatureSetSelector(name='group_one', sel_subset=['a', 'b', 'c'])
Pipeline(steps=[('featureunion',
FeatureUnion(transformer_list=[('featureunion',
FeatureUnion(transformer_list=[('pca',
PCA(n_components=0.93113403057))])),
('passthrough',
Passthrough())]))])FeatureUnion(transformer_list=[('featureunion',
FeatureUnion(transformer_list=[('pca',
PCA(n_components=0.93113403057))])),
('passthrough', Passthrough())])PCA(n_components=0.93113403057)
Passthrough()
FeatureSetSelector(name='group_four', sel_subset=['j', 'k', 'l'])
Pipeline(steps=[('featureunion-1',
FeatureUnion(transformer_list=[('featureunion',
FeatureUnion(transformer_list=[('binarizer',
Binarizer(threshold=0.5396272782675))])),
('passthrough',
Passthrough())])),
('featureunion-2',
FeatureUnion(transformer_list=[('featureunion',
FeatureUnion(transformer_list=[('quantiletransformer',
QuantileTransformer(n_quantiles=87)),
('columnonehotencoder',
ColumnOneHotEncoder())])),
('passthrough',
Passthrough())]))])FeatureUnion(transformer_list=[('featureunion',
FeatureUnion(transformer_list=[('binarizer',
Binarizer(threshold=0.5396272782675))])),
('passthrough', Passthrough())])Binarizer(threshold=0.5396272782675)
Passthrough()
FeatureUnion(transformer_list=[('featureunion',
FeatureUnion(transformer_list=[('quantiletransformer',
QuantileTransformer(n_quantiles=87)),
('columnonehotencoder',
ColumnOneHotEncoder())])),
('passthrough', Passthrough())])QuantileTransformer(n_quantiles=87)
ColumnOneHotEncoder()
Passthrough()
RandomForestClassifier(class_weight='balanced', criterion='entropy',
max_features=0.021545996678, min_samples_leaf=11,
n_estimators=128)其他示例¶
字典¶
In [21]:
Copied!
import tpot2
import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression
import sklearn
subsets = { "group_one" : ['a','b','c'],
"group_two" : ['d','e','f'],
"group_three" : ['g','h','i'],
}
fss_search_space = tpot2.search_spaces.nodes.FSSNode(subsets=subsets)
selector = fss_search_space.generate(rng=1).export_pipeline()
selector.set_output(transform="pandas")
selector.fit(X_train)
selector.transform(X_train)
import tpot2
import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression
import sklearn
subsets = { "group_one" : ['a','b','c'],
"group_two" : ['d','e','f'],
"group_three" : ['g','h','i'],
}
fss_search_space = tpot2.search_spaces.nodes.FSSNode(subsets=subsets)
selector = fss_search_space.generate(rng=1).export_pipeline()
selector.set_output(transform="pandas")
selector.fit(X_train)
selector.transform(X_train)
Out[21]:
| d | e | f | |
|---|---|---|---|
| 28 | -2.393671 | 2.653494 | 1.336840 |
| 540 | -1.598037 | -2.639941 | -1.787062 |
| 980 | -1.562249 | 1.573867 | -0.135207 |
| 812 | 0.084835 | 1.809188 | -1.525609 |
| 117 | 0.647414 | 1.437139 | 1.873279 |
| ... | ... | ... | ... |
| 630 | 0.102721 | 0.463829 | -0.220689 |
| 963 | -0.530709 | 0.353686 | 0.621369 |
| 943 | 3.850193 | 0.948248 | -2.042764 |
| 930 | 1.051634 | 1.240570 | -1.477092 |
| 116 | -0.126476 | -1.599799 | -0.610169 |
750 行 × 3 列
列表¶
In [22]:
Copied!
import tpot2
import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression
import sklearn
subsets = [['a','b','c'],['d','e','f'],['g','h','i']]
fss_search_space = tpot2.search_spaces.nodes.FSSNode(subsets=subsets)
selector = fss_search_space.generate(rng=1).export_pipeline()
selector.set_output(transform="pandas")
selector.fit(X_train)
selector.transform(X_train)
import tpot2
import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression
import sklearn
subsets = [['a','b','c'],['d','e','f'],['g','h','i']]
fss_search_space = tpot2.search_spaces.nodes.FSSNode(subsets=subsets)
selector = fss_search_space.generate(rng=1).export_pipeline()
selector.set_output(transform="pandas")
selector.fit(X_train)
selector.transform(X_train)
Out[22]:
| d | e | f | |
|---|---|---|---|
| 28 | -2.393671 | 2.653494 | 1.336840 |
| 540 | -1.598037 | -2.639941 | -1.787062 |
| 980 | -1.562249 | 1.573867 | -0.135207 |
| 812 | 0.084835 | 1.809188 | -1.525609 |
| 117 | 0.647414 | 1.437139 | 1.873279 |
| ... | ... | ... | ... |
| 630 | 0.102721 | 0.463829 | -0.220689 |
| 963 | -0.530709 | 0.353686 | 0.621369 |
| 943 | 3.850193 | 0.948248 | -2.042764 |
| 930 | 1.051634 | 1.240570 | -1.477092 |
| 116 | -0.126476 | -1.599799 | -0.610169 |
750 行 × 3 列
CSV 文件¶
注意：注意csv文件中的空格！
In [23]:
Copied!
import tpot2
import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression
import sklearn
subsets = 'simple_fss.csv'
'''
# simple_fss.csv
one,a,b,c
two,d,e,f
three,g,h,i
'''
fss_search_space = tpot2.search_spaces.nodes.FSSNode(subsets=subsets)
selector = fss_search_space.generate(rng=1).export_pipeline()
selector.set_output(transform="pandas")
selector.fit(X_train)
selector.transform(X_train)
import tpot2
import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression
import sklearn
subsets = 'simple_fss.csv'
'''
# simple_fss.csv
one,a,b,c
two,d,e,f
three,g,h,i
'''
fss_search_space = tpot2.search_spaces.nodes.FSSNode(subsets=subsets)
selector = fss_search_space.generate(rng=1).export_pipeline()
selector.set_output(transform="pandas")
selector.fit(X_train)
selector.transform(X_train)
Out[23]:
| d | e | f | |
|---|---|---|---|
| 28 | -2.393671 | 2.653494 | 1.336840 |
| 540 | -1.598037 | -2.639941 | -1.787062 |
| 980 | -1.562249 | 1.573867 | -0.135207 |
| 812 | 0.084835 | 1.809188 | -1.525609 |
| 117 | 0.647414 | 1.437139 | 1.873279 |
| ... | ... | ... | ... |
| 630 | 0.102721 | 0.463829 | -0.220689 |
| 963 | -0.530709 | 0.353686 | 0.621369 |
| 943 | 3.850193 | 0.948248 | -2.042764 |
| 930 | 1.051634 | 1.240570 | -1.477092 |
| 116 | -0.126476 | -1.599799 | -0.610169 |
750 行 × 3 列
以上所有内容在使用numpy数据时都是相同的，但列名被替换为整数索引。
In [24]:
Copied!
import tpot2
import sklearn.datasets
from sklearn.linear_model import LogisticRegression
import numpy as np
import pandas as pd
n_features = 6
X, y = sklearn.datasets.make_classification(n_samples=1000, n_features=n_features, n_informative=6, n_redundant=0, n_repeated=0, n_classes=2, n_clusters_per_class=2, weights=None, flip_y=0.01, class_sep=1.0, hypercube=True, shift=0.0, scale=1.0, shuffle=True, random_state=None)
X = np.hstack([X, np.random.rand(X.shape[0],3)]) #add three uninformative features
X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, train_size=0.75, test_size=0.25)
print(X)
import tpot2
import sklearn.datasets
from sklearn.linear_model import LogisticRegression
import numpy as np
import pandas as pd
n_features = 6
X, y = sklearn.datasets.make_classification(n_samples=1000, n_features=n_features, n_informative=6, n_redundant=0, n_repeated=0, n_classes=2, n_clusters_per_class=2, weights=None, flip_y=0.01, class_sep=1.0, hypercube=True, shift=0.0, scale=1.0, shuffle=True, random_state=None)
X = np.hstack([X, np.random.rand(X.shape[0],3)]) #add three uninformative features
X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, train_size=0.75, test_size=0.25)
print(X)
[[-0.43714166 -0.50887207 -3.17945595 ... 0.07671291 0.47607558 0.89683945] [-2.83836404 -0.22115893 -0.07445108 ... 0.03073931 0.2766683 0.36285899] [-2.28029617 -1.38851427 -3.22134569 ... 0.92830528 0.59176052 0.18041296] ... [ 0.61359823 -0.41893724 -2.9625971 ... 0.75602013 0.52478388 0.69249969] [-2.27709727 2.99680411 0.70411587 ... 0.02910316 0.93519319 0.0034257 ] [-1.59654364 -0.53352175 -0.50919438 ... 0.63719765 0.47591644 0.84288743]]
In [25]:
Copied!
import tpot2
import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression
import sklearn
subsets = { "group_one" : [0,1,2],
"group_two" : [3,4,5],
"group_three" : [6,7,8],
}
fss_search_space = tpot2.search_spaces.nodes.FSSNode(subsets=subsets)
selector = fss_search_space.generate(rng=1).export_pipeline()
selector.fit(X_train)
selector.transform(X_train)
import tpot2
import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression
import sklearn
subsets = { "group_one" : [0,1,2],
"group_two" : [3,4,5],
"group_three" : [6,7,8],
}
fss_search_space = tpot2.search_spaces.nodes.FSSNode(subsets=subsets)
selector = fss_search_space.generate(rng=1).export_pipeline()
selector.fit(X_train)
selector.transform(X_train)
Out[25]:
array([[-2.15063999, -0.84591563, -0.66736542],
[-0.95324351, 2.00496434, 1.22398102],
[-0.08542414, -0.26901573, -3.67530636],
...,
[ 0.48872267, -0.87071824, 1.60102349],
[-4.45746257, -2.41209776, 0.42331464],
[-0.72541871, -0.02783289, -1.98627911]])