预测表中的多列（多标签预测）¶
在多标签预测中，我们希望基于表中剩余列的值来预测表的多个列（即标签）。这里我们介绍一种使用AutoGluon的简单策略，即为每个被预测的列维护一个单独的TabularPredictor对象。通过在标签上施加顺序并允许每个标签的TabularPredictor依赖于顺序中较早出现的标签的预测值，可以在预测中考虑标签之间的相关性。
多标签预测器类¶
我们首先定义一个自定义的MultilabelPredictor类来管理一组TabularPredictor对象，每个标签对应一个。你可以像使用单个TabularPredictor一样使用MultilabelPredictor，只是它操作的是多个标签而不是一个。
from autogluon.tabular import TabularDataset, TabularPredictor
from autogluon.common.utils.utils import setup_outputdir
from autogluon.core.utils.loaders import load_pkl
from autogluon.core.utils.savers import save_pkl
import os.path
class MultilabelPredictor:
""" Tabular Predictor for predicting multiple columns in table.
Creates multiple TabularPredictor objects which you can also use individually.
You can access the TabularPredictor for a particular label via: `multilabel_predictor.get_predictor(label_i)`
Parameters
----------
labels : List[str]
The ith element of this list is the column (i.e. `label`) predicted by the ith TabularPredictor stored in this object.
path : str, default = None
Path to directory where models and intermediate outputs should be saved.
If unspecified, a time-stamped folder called "AutogluonModels/ag-[TIMESTAMP]" will be created in the working directory to store all models.
Note: To call `fit()` twice and save all results of each fit, you must specify different `path` locations or don't specify `path` at all.
Otherwise files from first `fit()` will be overwritten by second `fit()`.
Caution: when predicting many labels, this directory may grow large as it needs to store many TabularPredictors.
problem_types : List[str], default = None
The ith element is the `problem_type` for the ith TabularPredictor stored in this object.
eval_metrics : List[str], default = None
The ith element is the `eval_metric` for the ith TabularPredictor stored in this object.
consider_labels_correlation : bool, default = True
Whether the predictions of multiple labels should account for label correlations or predict each label independently of the others.
If True, the ordering of `labels` may affect resulting accuracy as each label is predicted conditional on the previous labels appearing earlier in this list (i.e. in an auto-regressive fashion).
Set to False if during inference you may want to individually use just the ith TabularPredictor without predicting all the other labels.
kwargs :
Arguments passed into the initialization of each TabularPredictor.
"""
multi_predictor_file = 'multilabel_predictor.pkl'
def __init__(self, labels, path=None, problem_types=None, eval_metrics=None, consider_labels_correlation=True, **kwargs):
if len(labels) < 2:
raise ValueError("MultilabelPredictor is only intended for predicting MULTIPLE labels (columns), use TabularPredictor for predicting one label (column).")
if (problem_types is not None) and (len(problem_types) != len(labels)):
raise ValueError("If provided, `problem_types` must have same length as `labels`")
if (eval_metrics is not None) and (len(eval_metrics) != len(labels)):
raise ValueError("If provided, `eval_metrics` must have same length as `labels`")
self.path = setup_outputdir(path, warn_if_exist=False)
self.labels = labels
self.consider_labels_correlation = consider_labels_correlation
self.predictors = {} # key = label, value = TabularPredictor or str path to the TabularPredictor for this label
if eval_metrics is None:
self.eval_metrics = {}
else:
self.eval_metrics = {labels[i] : eval_metrics[i] for i in range(len(labels))}
problem_type = None
eval_metric = None
for i in range(len(labels)):
label = labels[i]
path_i = os.path.join(self.path, "Predictor_" + str(label))
if problem_types is not None:
problem_type = problem_types[i]
if eval_metrics is not None:
eval_metric = eval_metrics[i]
self.predictors[label] = TabularPredictor(label=label, problem_type=problem_type, eval_metric=eval_metric, path=path_i, **kwargs)
def fit(self, train_data, tuning_data=None, **kwargs):
""" Fits a separate TabularPredictor to predict each of the labels.
Parameters
----------
train_data, tuning_data : str or pd.DataFrame
See documentation for `TabularPredictor.fit()`.
kwargs :
Arguments passed into the `fit()` call for each TabularPredictor.
"""
if isinstance(train_data, str):
train_data = TabularDataset(train_data)
if tuning_data is not None and isinstance(tuning_data, str):
tuning_data = TabularDataset(tuning_data)
train_data_og = train_data.copy()
if tuning_data is not None:
tuning_data_og = tuning_data.copy()
else:
tuning_data_og = None
save_metrics = len(self.eval_metrics) == 0
for i in range(len(self.labels)):
label = self.labels[i]
predictor = self.get_predictor(label)
if not self.consider_labels_correlation:
labels_to_drop = [l for l in self.labels if l != label]
else:
labels_to_drop = [self.labels[j] for j in range(i+1, len(self.labels))]
train_data = train_data_og.drop(labels_to_drop, axis=1)
if tuning_data is not None:
tuning_data = tuning_data_og.drop(labels_to_drop, axis=1)
print(f"Fitting TabularPredictor for label: {label} ...")
predictor.fit(train_data=train_data, tuning_data=tuning_data, **kwargs)
self.predictors[label] = predictor.path
if save_metrics:
self.eval_metrics[label] = predictor.eval_metric
self.save()
def predict(self, data, **kwargs):
""" Returns DataFrame with label columns containing predictions for each label.
Parameters
----------
data : str or autogluon.tabular.TabularDataset or pd.DataFrame
Data to make predictions for. If label columns are present in this data, they will be ignored. See documentation for `TabularPredictor.predict()`.
kwargs :
Arguments passed into the predict() call for each TabularPredictor.
"""
return self._predict(data, as_proba=False, **kwargs)
def predict_proba(self, data, **kwargs):
""" Returns dict where each key is a label and the corresponding value is the `predict_proba()` output for just that label.
Parameters
----------
data : str or autogluon.tabular.TabularDataset or pd.DataFrame
Data to make predictions for. See documentation for `TabularPredictor.predict()` and `TabularPredictor.predict_proba()`.
kwargs :
Arguments passed into the `predict_proba()` call for each TabularPredictor (also passed into a `predict()` call).
"""
return self._predict(data, as_proba=True, **kwargs)
def evaluate(self, data, **kwargs):
""" Returns dict where each key is a label and the corresponding value is the `evaluate()` output for just that label.
Parameters
----------
data : str or autogluon.tabular.TabularDataset or pd.DataFrame
Data to evalate predictions of all labels for, must contain all labels as columns. See documentation for `TabularPredictor.evaluate()`.
kwargs :
Arguments passed into the `evaluate()` call for each TabularPredictor (also passed into the `predict()` call).
"""
data = self._get_data(data)
eval_dict = {}
for label in self.labels:
print(f"Evaluating TabularPredictor for label: {label} ...")
predictor = self.get_predictor(label)
eval_dict[label] = predictor.evaluate(data, **kwargs)
if self.consider_labels_correlation:
data[label] = predictor.predict(data, **kwargs)
return eval_dict
def save(self):
""" Save MultilabelPredictor to disk. """
for label in self.labels:
if not isinstance(self.predictors[label], str):
self.predictors[label] = self.predictors[label].path
save_pkl.save(path=os.path.join(self.path, self.multi_predictor_file), object=self)
print(f"MultilabelPredictor saved to disk. Load with: MultilabelPredictor.load('{self.path}')")
@classmethod
def load(cls, path):
""" Load MultilabelPredictor from disk `path` previously specified when creating this MultilabelPredictor. """
path = os.path.expanduser(path)
return load_pkl.load(path=os.path.join(path, cls.multi_predictor_file))
def get_predictor(self, label):
""" Returns TabularPredictor which is used to predict this label. """
predictor = self.predictors[label]
if isinstance(predictor, str):
return TabularPredictor.load(path=predictor)
return predictor
def _get_data(self, data):
if isinstance(data, str):
return TabularDataset(data)
return data.copy()
def _predict(self, data, as_proba=False, **kwargs):
data = self._get_data(data)
if as_proba:
predproba_dict = {}
for label in self.labels:
print(f"Predicting with TabularPredictor for label: {label} ...")
predictor = self.get_predictor(label)
if as_proba:
predproba_dict[label] = predictor.predict_proba(data, as_multiclass=True, **kwargs)
data[label] = predictor.predict(data, **kwargs)
if not as_proba:
return data[self.labels]
else:
return predproba_dict
Training¶
现在让我们应用我们的多标签预测器来预测数据表中的多个列。我们首先训练模型来预测每个标签。
train_data = TabularDataset('https://autogluon.s3.amazonaws.com/datasets/Inc/train.csv')
subsample_size = 500 # subsample subset of data for faster demo, try setting this to much larger values
train_data = train_data.sample(n=subsample_size, random_state=0)
train_data.head()
| age | workclass | fnlwgt | education | education-num | marital-status | occupation | relationship | race | sex | capital-gain | capital-loss | hours-per-week | native-country | class | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 6118 | 51 | Private | 39264 | Some-college | 10 | Married-civ-spouse | Exec-managerial | Wife | White | Female | 0 | 0 | 40 | United-States | >50K |
| 23204 | 58 | Private | 51662 | 10th | 6 | Married-civ-spouse | Other-service | Wife | White | Female | 0 | 0 | 8 | United-States | <=50K |
| 29590 | 40 | Private | 326310 | Some-college | 10 | Married-civ-spouse | Craft-repair | Husband | White | Male | 0 | 0 | 44 | United-States | <=50K |
| 18116 | 37 | Private | 222450 | HS-grad | 9 | Never-married | Sales | Not-in-family | White | Male | 0 | 2339 | 40 | El-Salvador | <=50K |
| 33964 | 62 | Private | 109190 | Bachelors | 13 | Married-civ-spouse | Exec-managerial | Husband | White | Male | 15024 | 0 | 40 | United-States | >50K |
labels = ['education-num','education','class'] # which columns to predict based on the others
problem_types = ['regression','multiclass','binary'] # type of each prediction problem (optional)
eval_metrics = ['mean_absolute_error','accuracy','accuracy'] # metrics used to evaluate predictions for each label (optional)
save_path = 'agModels-predictEducationClass' # specifies folder to store trained models (optional)
time_limit = 5 # how many seconds to train the TabularPredictor for each label, set much larger in your applications!
multi_predictor = MultilabelPredictor(labels=labels, problem_types=problem_types, eval_metrics=eval_metrics, path=save_path)
multi_predictor.fit(train_data, time_limit=time_limit)
Fitting TabularPredictor for label: education-num ...
Fitting TabularPredictor for label: education ...
Fitting TabularPredictor for label: class ...
MultilabelPredictor saved to disk. Load with: MultilabelPredictor.load('/home/ci/autogluon/docs/tutorials/tabular/advanced/agModels-predictEducationClass')
Verbosity: 2 (Standard Logging)
=================== System Info ===================
AutoGluon Version: 1.2b20241127
Python Version: 3.11.9
Operating System: Linux
Platform Machine: x86_64
Platform Version: #1 SMP Tue Sep 24 10:00:37 UTC 2024
CPU Count: 8
Memory Avail: 28.81 GB / 30.95 GB (93.1%)
Disk Space Avail: 213.43 GB / 255.99 GB (83.4%)
===================================================
No presets specified! To achieve strong results with AutoGluon, it is recommended to use the available presets. Defaulting to `'medium'`...
Recommended Presets (For more details refer to https://auto.gluon.ai/stable/tutorials/tabular/tabular-essentials.html#presets):
presets='experimental' : New in v1.2: Pre-trained foundation model + parallel fits. The absolute best accuracy without consideration for inference speed. Does not support GPU.
presets='best' : Maximize accuracy. Recommended for most users. Use in competitions and benchmarks.
presets='high' : Strong accuracy with fast inference speed.
presets='good' : Good accuracy with very fast inference speed.
presets='medium' : Fast training time, ideal for initial prototyping.
Beginning AutoGluon training ... Time limit = 5s
AutoGluon will save models to "/home/ci/autogluon/docs/tutorials/tabular/advanced/agModels-predictEducationClass/Predictor_education-num"
Train Data Rows: 500
Train Data Columns: 12
Label Column: education-num
Problem Type: regression
Preprocessing data ...
Using Feature Generators to preprocess the data ...
Fitting AutoMLPipelineFeatureGenerator...
Available Memory: 29493.55 MB
Train Data (Original) Memory Usage: 0.24 MB (0.0% of available memory)
Inferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.
Stage 1 Generators:
Fitting AsTypeFeatureGenerator...
Note: Converting 1 features to boolean dtype as they only contain 2 unique values.
Stage 2 Generators:
Fitting FillNaFeatureGenerator...
Stage 3 Generators:
Fitting IdentityFeatureGenerator...
Fitting CategoryFeatureGenerator...
Fitting CategoryMemoryMinimizeFeatureGenerator...
Stage 4 Generators:
Fitting DropUniqueFeatureGenerator...
Stage 5 Generators:
Fitting DropDuplicatesFeatureGenerator...
Types of features in original data (raw dtype, special dtypes):
('int', []) : 5 | ['age', 'fnlwgt', 'capital-gain', 'capital-loss', 'hours-per-week']
('object', []) : 7 | ['workclass', 'marital-status', 'occupation', 'relationship', 'race', ...]
Types of features in processed data (raw dtype, special dtypes):
('category', []) : 6 | ['workclass', 'marital-status', 'occupation', 'relationship', 'race', ...]
('int', []) : 5 | ['age', 'fnlwgt', 'capital-gain', 'capital-loss', 'hours-per-week']
('int', ['bool']) : 1 | ['sex']
0.1s = Fit runtime
12 features in original data used to generate 12 features in processed data.
Train Data (Processed) Memory Usage: 0.03 MB (0.0% of available memory)
Data preprocessing and feature engineering runtime = 0.07s ...
AutoGluon will gauge predictive performance using evaluation metric: 'mean_absolute_error'
This metric's sign has been flipped to adhere to being higher_is_better. The metric score can be multiplied by -1 to get the metric value.
To change this, specify the eval_metric parameter of Predictor()
Automatically generating train/validation split with holdout_frac=0.2, Train Rows: 400, Val Rows: 100
User-specified model hyperparameters to be fit:
{
'NN_TORCH': [{}],
'GBM': [{'extra_trees': True, 'ag_args': {'name_suffix': 'XT'}}, {}, {'learning_rate': 0.03, 'num_leaves': 128, 'feature_fraction': 0.9, 'min_data_in_leaf': 3, 'ag_args': {'name_suffix': 'Large', 'priority': 0, 'hyperparameter_tune_kwargs': None}}],
'CAT': [{}],
'XGB': [{}],
'FASTAI': [{}],
'RF': [{'criterion': 'gini', 'ag_args': {'name_suffix': 'Gini', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'entropy', 'ag_args': {'name_suffix': 'Entr', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'squared_error', 'ag_args': {'name_suffix': 'MSE', 'problem_types': ['regression', 'quantile']}}],
'XT': [{'criterion': 'gini', 'ag_args': {'name_suffix': 'Gini', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'entropy', 'ag_args': {'name_suffix': 'Entr', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'squared_error', 'ag_args': {'name_suffix': 'MSE', 'problem_types': ['regression', 'quantile']}}],
'KNN': [{'weights': 'uniform', 'ag_args': {'name_suffix': 'Unif'}}, {'weights': 'distance', 'ag_args': {'name_suffix': 'Dist'}}],
}
Fitting 11 L1 models, fit_strategy="sequential" ...
Fitting model: KNeighborsUnif ... Training model for up to 4.93s of the 4.93s of remaining time.
-2.086 = Validation score (-mean_absolute_error)
0.04s = Training runtime
0.01s = Validation runtime
Fitting model: KNeighborsDist ... Training model for up to 4.87s of the 4.87s of remaining time.
-2.1856 = Validation score (-mean_absolute_error)
0.01s = Training runtime
0.01s = Validation runtime
Fitting model: LightGBMXT ... Training model for up to 4.85s of the 4.85s of remaining time.
-1.7808 = Validation score (-mean_absolute_error)
0.3s = Training runtime
0.0s = Validation runtime
Fitting model: LightGBM ... Training model for up to 4.54s of the 4.54s of remaining time.
-1.7854 = Validation score (-mean_absolute_error)
0.22s = Training runtime
0.0s = Validation runtime
Fitting model: RandomForestMSE ... Training model for up to 4.31s of the 4.31s of remaining time.
-1.7082 = Validation score (-mean_absolute_error)
0.54s = Training runtime
0.05s = Validation runtime
Fitting model: CatBoost ... Training model for up to 3.70s of the 3.70s of remaining time.
-1.7377 = Validation score (-mean_absolute_error)
1.07s = Training runtime
0.0s = Validation runtime
Fitting model: ExtraTreesMSE ... Training model for up to 2.62s of the 2.62s of remaining time.
-1.8193 = Validation score (-mean_absolute_error)
0.46s = Training runtime
0.05s = Validation runtime
Fitting model: NeuralNetFastAI ... Training model for up to 2.11s of the 2.11s of remaining time.
-1.8891 = Validation score (-mean_absolute_error)
2.66s = Training runtime
0.01s = Validation runtime
Fitting model: WeightedEnsemble_L2 ... Training model for up to 4.93s of the -0.59s of remaining time.
Ensemble Weights: {'RandomForestMSE': 0.619, 'CatBoost': 0.238, 'LightGBMXT': 0.143}
-1.689 = Validation score (-mean_absolute_error)
0.05s = Training runtime
0.0s = Validation runtime
AutoGluon training complete, total runtime = 5.67s ... Best model: WeightedEnsemble_L2 | Estimated inference throughput: 1795.1 rows/s (100 batch size)
TabularPredictor saved. To load, use: predictor = TabularPredictor.load("/home/ci/autogluon/docs/tutorials/tabular/advanced/agModels-predictEducationClass/Predictor_education-num")
Verbosity: 2 (Standard Logging)
=================== System Info ===================
AutoGluon Version: 1.2b20241127
Python Version: 3.11.9
Operating System: Linux
Platform Machine: x86_64
Platform Version: #1 SMP Tue Sep 24 10:00:37 UTC 2024
CPU Count: 8
Memory Avail: 28.44 GB / 30.95 GB (91.9%)
Disk Space Avail: 213.41 GB / 255.99 GB (83.4%)
===================================================
No presets specified! To achieve strong results with AutoGluon, it is recommended to use the available presets. Defaulting to `'medium'`...
Recommended Presets (For more details refer to https://auto.gluon.ai/stable/tutorials/tabular/tabular-essentials.html#presets):
presets='experimental' : New in v1.2: Pre-trained foundation model + parallel fits. The absolute best accuracy without consideration for inference speed. Does not support GPU.
presets='best' : Maximize accuracy. Recommended for most users. Use in competitions and benchmarks.
presets='high' : Strong accuracy with fast inference speed.
presets='good' : Good accuracy with very fast inference speed.
presets='medium' : Fast training time, ideal for initial prototyping.
Beginning AutoGluon training ... Time limit = 5s
AutoGluon will save models to "/home/ci/autogluon/docs/tutorials/tabular/advanced/agModels-predictEducationClass/Predictor_education"
Train Data Rows: 500
Train Data Columns: 13
Label Column: education
Problem Type: multiclass
Preprocessing data ...
Warning: Some classes in the training set have fewer than 10 examples. AutoGluon will only keep 11 out of 15 classes for training and will not try to predict the rare classes. To keep more classes, increase the number of datapoints from these rare classes in the training data or reduce label_count_threshold.
Fraction of data from classes with at least 10 examples that will be kept for training models: 0.976
Train Data Class Count: 11
Using Feature Generators to preprocess the data ...
Fitting AutoMLPipelineFeatureGenerator...
Available Memory: 29119.79 MB
Train Data (Original) Memory Usage: 0.24 MB (0.0% of available memory)
Inferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.
Stage 1 Generators:
Fitting AsTypeFeatureGenerator...
Note: Converting 1 features to boolean dtype as they only contain 2 unique values.
Stage 2 Generators:
Fitting FillNaFeatureGenerator...
Stage 3 Generators:
Fitting IdentityFeatureGenerator...
Fitting CategoryFeatureGenerator...
Fitting CategoryMemoryMinimizeFeatureGenerator...
Stage 4 Generators:
Fitting DropUniqueFeatureGenerator...
Stage 5 Generators:
Fitting DropDuplicatesFeatureGenerator...
Types of features in original data (raw dtype, special dtypes):
('int', []) : 6 | ['age', 'fnlwgt', 'education-num', 'capital-gain', 'capital-loss', ...]
('object', []) : 7 | ['workclass', 'marital-status', 'occupation', 'relationship', 'race', ...]
Types of features in processed data (raw dtype, special dtypes):
('category', []) : 6 | ['workclass', 'marital-status', 'occupation', 'relationship', 'race', ...]
('int', []) : 6 | ['age', 'fnlwgt', 'education-num', 'capital-gain', 'capital-loss', ...]
('int', ['bool']) : 1 | ['sex']
0.1s = Fit runtime
13 features in original data used to generate 13 features in processed data.
Train Data (Processed) Memory Usage: 0.03 MB (0.0% of available memory)
Data preprocessing and feature engineering runtime = 0.09s ...
AutoGluon will gauge predictive performance using evaluation metric: 'accuracy'
To change this, specify the eval_metric parameter of Predictor()
Automatically generating train/validation split with holdout_frac=0.2, Train Rows: 390, Val Rows: 98
User-specified model hyperparameters to be fit:
{
'NN_TORCH': [{}],
'GBM': [{'extra_trees': True, 'ag_args': {'name_suffix': 'XT'}}, {}, {'learning_rate': 0.03, 'num_leaves': 128, 'feature_fraction': 0.9, 'min_data_in_leaf': 3, 'ag_args': {'name_suffix': 'Large', 'priority': 0, 'hyperparameter_tune_kwargs': None}}],
'CAT': [{}],
'XGB': [{}],
'FASTAI': [{}],
'RF': [{'criterion': 'gini', 'ag_args': {'name_suffix': 'Gini', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'entropy', 'ag_args': {'name_suffix': 'Entr', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'squared_error', 'ag_args': {'name_suffix': 'MSE', 'problem_types': ['regression', 'quantile']}}],
'XT': [{'criterion': 'gini', 'ag_args': {'name_suffix': 'Gini', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'entropy', 'ag_args': {'name_suffix': 'Entr', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'squared_error', 'ag_args': {'name_suffix': 'MSE', 'problem_types': ['regression', 'quantile']}}],
'KNN': [{'weights': 'uniform', 'ag_args': {'name_suffix': 'Unif'}}, {'weights': 'distance', 'ag_args': {'name_suffix': 'Dist'}}],
}
Fitting 13 L1 models, fit_strategy="sequential" ...
Fitting model: KNeighborsUnif ... Training model for up to 4.91s of the 4.91s of remaining time.
0.2653 = Validation score (accuracy)
0.01s = Training runtime
0.01s = Validation runtime
Fitting model: KNeighborsDist ... Training model for up to 4.88s of the 4.88s of remaining time.
0.2347 = Validation score (accuracy)
0.01s = Training runtime
0.01s = Validation runtime
Fitting model: NeuralNetFastAI ... Training model for up to 4.85s of the 4.85s of remaining time.
0.7653 = Validation score (accuracy)
0.5s = Training runtime
0.01s = Validation runtime
Fitting model: LightGBMXT ... Training model for up to 4.34s of the 4.34s of remaining time.
0.9694 = Validation score (accuracy)
0.69s = Training runtime
0.01s = Validation runtime
Fitting model: LightGBM ... Training model for up to 3.60s of the 3.60s of remaining time.
1.0 = Validation score (accuracy)
0.42s = Training runtime
0.0s = Validation runtime
Fitting model: RandomForestGini ... Training model for up to 3.17s of the 3.17s of remaining time.
0.9082 = Validation score (accuracy)
0.83s = Training runtime
0.06s = Validation runtime
Fitting model: RandomForestEntr ... Training model for up to 2.26s of the 2.26s of remaining time.
0.9082 = Validation score (accuracy)
0.87s = Training runtime
0.06s = Validation runtime
Fitting model: CatBoost ... Training model for up to 1.30s of the 1.30s of remaining time.
Ran out of time, early stopping on iteration 71.
0.8878 = Validation score (accuracy)
1.28s = Training runtime
0.0s = Validation runtime
Fitting model: WeightedEnsemble_L2 ... Training model for up to 4.91s of the 0.00s of remaining time.
Ensemble Weights: {'LightGBM': 1.0}
1.0 = Validation score (accuracy)
0.05s = Training runtime
0.0s = Validation runtime
AutoGluon training complete, total runtime = 5.1s ... Best model: WeightedEnsemble_L2 | Estimated inference throughput: 22572.3 rows/s (98 batch size)
TabularPredictor saved. To load, use: predictor = TabularPredictor.load("/home/ci/autogluon/docs/tutorials/tabular/advanced/agModels-predictEducationClass/Predictor_education")
Verbosity: 2 (Standard Logging)
=================== System Info ===================
AutoGluon Version: 1.2b20241127
Python Version: 3.11.9
Operating System: Linux
Platform Machine: x86_64
Platform Version: #1 SMP Tue Sep 24 10:00:37 UTC 2024
CPU Count: 8
Memory Avail: 28.35 GB / 30.95 GB (91.6%)
Disk Space Avail: 213.39 GB / 255.99 GB (83.4%)
===================================================
No presets specified! To achieve strong results with AutoGluon, it is recommended to use the available presets. Defaulting to `'medium'`...
Recommended Presets (For more details refer to https://auto.gluon.ai/stable/tutorials/tabular/tabular-essentials.html#presets):
presets='experimental' : New in v1.2: Pre-trained foundation model + parallel fits. The absolute best accuracy without consideration for inference speed. Does not support GPU.
presets='best' : Maximize accuracy. Recommended for most users. Use in competitions and benchmarks.
presets='high' : Strong accuracy with fast inference speed.
presets='good' : Good accuracy with very fast inference speed.
presets='medium' : Fast training time, ideal for initial prototyping.
Beginning AutoGluon training ... Time limit = 5s
AutoGluon will save models to "/home/ci/autogluon/docs/tutorials/tabular/advanced/agModels-predictEducationClass/Predictor_class"
Train Data Rows: 500
Train Data Columns: 14
Label Column: class
Problem Type: binary
Preprocessing data ...
Selected class <--> label mapping: class 1 = >50K, class 0 = <=50K
Note: For your binary classification, AutoGluon arbitrarily selected which label-value represents positive ( >50K) vs negative ( <=50K) class.
To explicitly set the positive_class, either rename classes to 1 and 0, or specify positive_class in Predictor init.
Using Feature Generators to preprocess the data ...
Fitting AutoMLPipelineFeatureGenerator...
Available Memory: 29035.54 MB
Train Data (Original) Memory Usage: 0.28 MB (0.0% of available memory)
Inferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.
Stage 1 Generators:
Fitting AsTypeFeatureGenerator...
Note: Converting 1 features to boolean dtype as they only contain 2 unique values.
Stage 2 Generators:
Fitting FillNaFeatureGenerator...
Stage 3 Generators:
Fitting IdentityFeatureGenerator...
Fitting CategoryFeatureGenerator...
Fitting CategoryMemoryMinimizeFeatureGenerator...
Stage 4 Generators:
Fitting DropUniqueFeatureGenerator...
Stage 5 Generators:
Fitting DropDuplicatesFeatureGenerator...
Types of features in original data (raw dtype, special dtypes):
('int', []) : 6 | ['age', 'fnlwgt', 'education-num', 'capital-gain', 'capital-loss', ...]
('object', []) : 8 | ['workclass', 'education', 'marital-status', 'occupation', 'relationship', ...]
Types of features in processed data (raw dtype, special dtypes):
('category', []) : 7 | ['workclass', 'education', 'marital-status', 'occupation', 'relationship', ...]
('int', []) : 6 | ['age', 'fnlwgt', 'education-num', 'capital-gain', 'capital-loss', ...]
('int', ['bool']) : 1 | ['sex']
0.1s = Fit runtime
14 features in original data used to generate 14 features in processed data.
Train Data (Processed) Memory Usage: 0.03 MB (0.0% of available memory)
Data preprocessing and feature engineering runtime = 0.07s ...
AutoGluon will gauge predictive performance using evaluation metric: 'accuracy'
To change this, specify the eval_metric parameter of Predictor()
Automatically generating train/validation split with holdout_frac=0.2, Train Rows: 400, Val Rows: 100
User-specified model hyperparameters to be fit:
{
'NN_TORCH': [{}],
'GBM': [{'extra_trees': True, 'ag_args': {'name_suffix': 'XT'}}, {}, {'learning_rate': 0.03, 'num_leaves': 128, 'feature_fraction': 0.9, 'min_data_in_leaf': 3, 'ag_args': {'name_suffix': 'Large', 'priority': 0, 'hyperparameter_tune_kwargs': None}}],
'CAT': [{}],
'XGB': [{}],
'FASTAI': [{}],
'RF': [{'criterion': 'gini', 'ag_args': {'name_suffix': 'Gini', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'entropy', 'ag_args': {'name_suffix': 'Entr', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'squared_error', 'ag_args': {'name_suffix': 'MSE', 'problem_types': ['regression', 'quantile']}}],
'XT': [{'criterion': 'gini', 'ag_args': {'name_suffix': 'Gini', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'entropy', 'ag_args': {'name_suffix': 'Entr', 'problem_types': ['binary', 'multiclass']}}, {'criterion': 'squared_error', 'ag_args': {'name_suffix': 'MSE', 'problem_types': ['regression', 'quantile']}}],
'KNN': [{'weights': 'uniform', 'ag_args': {'name_suffix': 'Unif'}}, {'weights': 'distance', 'ag_args': {'name_suffix': 'Dist'}}],
}
Fitting 13 L1 models, fit_strategy="sequential" ...
Fitting model: KNeighborsUnif ... Training model for up to 4.93s of the 4.93s of remaining time.
0.73 = Validation score (accuracy)
0.01s = Training runtime
0.01s = Validation runtime
Fitting model: KNeighborsDist ... Training model for up to 4.90s of the 4.90s of remaining time.
0.65 = Validation score (accuracy)
0.01s = Training runtime
0.01s = Validation runtime
Fitting model: LightGBMXT ... Training model for up to 4.87s of the 4.87s of remaining time.
0.83 = Validation score (accuracy)
0.19s = Training runtime
0.0s = Validation runtime
Fitting model: LightGBM ... Training model for up to 4.68s of the 4.68s of remaining time.
0.85 = Validation score (accuracy)
0.23s = Training runtime
0.0s = Validation runtime
Fitting model: RandomForestGini ... Training model for up to 4.44s of the 4.44s of remaining time.
0.84 = Validation score (accuracy)
0.56s = Training runtime
0.05s = Validation runtime
Fitting model: RandomForestEntr ... Training model for up to 3.82s of the 3.82s of remaining time.
0.83 = Validation score (accuracy)
0.5s = Training runtime
0.05s = Validation runtime
Fitting model: CatBoost ... Training model for up to 3.26s of the 3.25s of remaining time.
0.85 = Validation score (accuracy)
0.78s = Training runtime
0.0s = Validation runtime
Fitting model: ExtraTreesGini ... Training model for up to 2.47s of the 2.47s of remaining time.
0.82 = Validation score (accuracy)
0.5s = Training runtime
0.05s = Validation runtime
Fitting model: ExtraTreesEntr ... Training model for up to 1.91s of the 1.91s of remaining time.
0.81 = Validation score (accuracy)
0.52s = Training runtime
0.05s = Validation runtime
Fitting model: NeuralNetFastAI ... Training model for up to 1.33s of the 1.33s of remaining time.
0.84 = Validation score (accuracy)
0.49s = Training runtime
0.01s = Validation runtime
Fitting model: XGBoost ... Training model for up to 0.82s of the 0.81s of remaining time.
0.86 = Validation score (accuracy)
0.24s = Training runtime
0.01s = Validation runtime
Fitting model: NeuralNetTorch ... Training model for up to 0.56s of the 0.56s of remaining time.
Time limit exceeded... Skipping NeuralNetTorch.
Fitting model: WeightedEnsemble_L2 ... Training model for up to 4.93s of the -0.54s of remaining time.
Ensemble Weights: {'XGBoost': 1.0}
0.86 = Validation score (accuracy)
0.07s = Training runtime
0.0s = Validation runtime
AutoGluon training complete, total runtime = 5.66s ... Best model: WeightedEnsemble_L2 | Estimated inference throughput: 14402.5 rows/s (100 batch size)
Disabling decision threshold calibration for metric `accuracy` due to having fewer than 10000 rows of validation data for calibration, to avoid overfitting (100 rows).
`accuracy` is generally not improved through threshold calibration. Force calibration via specifying `calibrate_decision_threshold=True`.
TabularPredictor saved. To load, use: predictor = TabularPredictor.load("/home/ci/autogluon/docs/tutorials/tabular/advanced/agModels-predictEducationClass/Predictor_class")
推理与评估¶
训练后，您可以轻松使用MultilabelPredictor来预测新数据中的所有标签：
test_data = TabularDataset('https://autogluon.s3.amazonaws.com/datasets/Inc/test.csv')
test_data = test_data.sample(n=subsample_size, random_state=0)
test_data_nolab = test_data.drop(columns=labels) # unnecessary, just to demonstrate we're not cheating here
test_data_nolab.head()
Loaded data from: https://autogluon.s3.amazonaws.com/datasets/Inc/test.csv | Columns = 15 / 15 | Rows = 9769 -> 9769
| age | workclass | fnlwgt | marital-status | occupation | relationship | race | sex | capital-gain | capital-loss | hours-per-week | native-country | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 5454 | 41 | Self-emp-not-inc | 408498 | Married-civ-spouse | Exec-managerial | Husband | White | Male | 0 | 0 | 50 | United-States |
| 6111 | 39 | Private | 746786 | Married-civ-spouse | Prof-specialty | Husband | White | Male | 0 | 0 | 55 | United-States |
| 5282 | 50 | Private | 62593 | Married-civ-spouse | Farming-fishing | Husband | Asian-Pac-Islander | Male | 0 | 0 | 40 | United-States |
| 3046 | 31 | Private | 248178 | Married-civ-spouse | Other-service | Husband | Black | Male | 0 | 0 | 35 | United-States |
| 2162 | 43 | State-gov | 52849 | Married-civ-spouse | Prof-specialty | Husband | White | Male | 0 | 0 | 40 | United-States |
multi_predictor = MultilabelPredictor.load(save_path) # unnecessary, just demonstrates how to load previously-trained multilabel predictor from file
predictions = multi_predictor.predict(test_data_nolab)
print("Predictions: \n", predictions)
Predicting with TabularPredictor for label: education-num ...
Predicting with TabularPredictor for label: education ...
Predicting with TabularPredictor for label: class ...
Predictions:
education-num education class
5454 10.934928 Some-college >50K
6111 13.357303 Bachelors >50K
5282 9.274375 HS-grad >50K
3046 9.487353 HS-grad <=50K
2162 12.900776 HS-grad >50K
... ... ... ...
6965 10.327562 Some-college >50K
4762 9.263704 HS-grad <=50K
234 10.478157 Some-college <=50K
6291 10.424629 Some-college <=50K
9575 9.883894 HS-grad >50K
[500 rows x 3 columns]
如果我们的新数据包含真实标签，我们也可以轻松评估我们预测的性能：
evaluations = multi_predictor.evaluate(test_data)
print(evaluations)
print("Evaluated using metrics:", multi_predictor.eval_metrics)
Evaluating TabularPredictor for label: education-num ...
Evaluating TabularPredictor for label: education ...
Evaluating TabularPredictor for label: class ...
{'education-num': {'mean_absolute_error': -1.6707148962020875, 'root_mean_squared_error': -2.26061055933517, 'mean_squared_error': -5.110360100977671, 'r2': 0.33920854330062866, 'pearsonr': 0.5997606498369555, 'median_absolute_error': -1.2628905773162842}, 'education': {'accuracy': 0.234, 'balanced_accuracy': 0.08531745398183771, 'mcc': 0.046833847121627255}, 'class': {'accuracy': 0.81, 'balanced_accuracy': 0.7076307486575149, 'mcc': 0.465025246821389, 'roc_auc': 0.847782026369126, 'f1': 0.5739910313901345, 'precision': 0.6808510638297872, 'recall': 0.49612403100775193}}
Evaluated using metrics: {'education-num': 'mean_absolute_error', 'education': 'accuracy', 'class': 'accuracy'}
访问单标签的TabularPredictor¶
我们也可以直接使用TabularPredictor来处理任何一个标签，如下所示。然而，如果您计划稍后使用单独的TabularPredictor来预测单个标签，而不是由MultilabelPredictor预测的所有标签，我们建议您在训练前设置consider_labels_correlation=False。
predictor_class = multi_predictor.get_predictor('class')
predictor_class.leaderboard()
| model | score_val | eval_metric | pred_time_val | fit_time | pred_time_val_marginal | fit_time_marginal | stack_level | can_infer | fit_order | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | XGBoost | 0.86 | accuracy | 0.006135 | 0.236339 | 0.006135 | 0.236339 | 1 | True | 11 |
| 1 | WeightedEnsemble_L2 | 0.86 | accuracy | 0.006943 | 0.305590 | 0.000808 | 0.069250 | 2 | True | 12 |
| 2 | LightGBM | 0.85 | accuracy | 0.003359 | 0.225952 | 0.003359 | 0.225952 | 1 | True | 4 |
| 3 | CatBoost | 0.85 | accuracy | 0.003616 | 0.776053 | 0.003616 | 0.776053 | 1 | True | 7 |
| 4 | NeuralNetFastAI | 0.84 | accuracy | 0.010530 | 0.493559 | 0.010530 | 0.493559 | 1 | True | 10 |
| 5 | RandomForestGini | 0.84 | accuracy | 0.047111 | 0.560902 | 0.047111 | 0.560902 | 1 | True | 5 |
| 6 | LightGBMXT | 0.83 | accuracy | 0.002942 | 0.188005 | 0.002942 | 0.188005 | 1 | True | 3 |
| 7 | RandomForestEntr | 0.83 | accuracy | 0.047146 | 0.504455 | 0.047146 | 0.504455 | 1 | True | 6 |
| 8 | ExtraTreesGini | 0.82 | accuracy | 0.046722 | 0.504215 | 0.046722 | 0.504215 | 1 | True | 8 |
| 9 | ExtraTreesEntr | 0.81 | accuracy | 0.046527 | 0.515158 | 0.046527 | 0.515158 | 1 | True | 9 |
| 10 | KNeighborsUnif | 0.73 | accuracy | 0.013555 | 0.010671 | 0.013555 | 0.010671 | 1 | True | 1 |
| 11 | KNeighborsDist | 0.65 | accuracy | 0.013182 | 0.008994 | 0.013182 | 0.008994 | 1 | True | 2 |
提示¶
为了获得最佳预测，通常应将以下参数添加到MultilabelPredictor.fit()中：
指定
eval_metrics为您将用于评估每个标签预测的指标指定
presets='best_quality'以告诉AutoGluon您更关心预测性能而不是延迟/内存使用，这将在预测每个标签时使用堆叠集成。
如果您发现使用了过多的内存/磁盘，请尝试使用在“深入教程中的如果您遇到内存问题”或“如果您遇到磁盘空间问题”下讨论的额外参数调用MultilabelPredictor.fit()。
如果您发现推理速度太慢，可以尝试在“深入教程中的加速推理”中讨论的策略。
特别是，只需在MultilabelPredictor.fit()中指定以下预设：presets = ['good_quality', 'optimize_for_deployment']