使用AutoMM进行持续训练¶
持续训练为机器学习模型提供了一种随时间改进其性能的方法。它使模型能够基于先前获得的知识进行构建，从而提高准确性，促进跨任务的知识转移，并节省计算资源。在本教程中，我们将展示使用AutoMM进行持续训练的三种用例。
用例1：通过额外数据或训练时间扩展训练¶
有时，模型可能会受益于更多的训练周期或额外的训练时间，以防欠拟合。使用AutoMM，您可以轻松延长模型的训练时间，而无需从头开始。
此外，通常还需要将更多数据整合到您的模型中。如果是一个多类问题，AutoMM 允许您继续使用相同问题类型和相同类别的数据进行训练。这种灵活性使得随着数据的增长，改进和调整您的模型变得容易。
我们使用斯坦福情感树库 (SST)数据集作为示例。它包含电影评论及其相关的情感。给定一个新的电影评论，目标是预测文本中反映的情感（在这种情况下是一个二元分类，如果评论传达积极意见则标记为1，否则标记为0）。让我们首先加载并查看数据，注意标签存储在名为label的列中。
from autogluon.core.utils.loaders import load_pd
train_data = load_pd.load("https://autogluon-text.s3-accelerate.amazonaws.com/glue/sst/train.parquet")
test_data = load_pd.load("https://autogluon-text.s3-accelerate.amazonaws.com/glue/sst/dev.parquet")
subsample_size = 1000 # subsample data for faster demo, try setting this to larger values
train_data_1 = train_data.sample(n=subsample_size, random_state=0)
train_data_1.head(10)
| sentence | label | |
|---|---|---|
| 43787 | very pleasing at its best moments | 1 |
| 16159 | , american chai is enough to make you put away... | 0 |
| 59015 | too much like an infomercial for ram dass 's l... | 0 |
| 5108 | a stirring visual sequence | 1 |
| 67052 | cool visual backmasking | 1 |
| 35938 | hard ground | 0 |
| 49879 | the striking , quietly vulnerable personality ... | 1 |
| 51591 | pan nalin 's exposition is beautiful and myste... | 1 |
| 56780 | wonderfully loopy | 1 |
| 28518 | most beautiful , evocative | 1 |
现在让我们训练模型。为了确保本教程快速运行，我们仅使用1000个训练示例的子集调用fit()，并将其运行时间限制在大约1分钟。为了在您的应用程序中获得合理的性能，建议您设置更长的时间限制（例如1小时），或者根本不指定时间限制（time_limit=None）。
from autogluon.multimodal import MultiModalPredictor
import uuid
model_path = f"./tmp/{uuid.uuid4().hex}-automm_sst"
predictor = MultiModalPredictor(label="label", eval_metric="acc", path=model_path)
predictor.fit(train_data_1, time_limit=60)
/home/ci/opt/venv/lib/python3.11/site-packages/mmengine/optim/optimizer/zero_optimizer.py:11: DeprecationWarning: `TorchScript` support for functional optimizers is deprecated and will be removed in a future PyTorch release. Consider using the `torch.compile` optimizer instead.
from torch.distributed.optim import \
=================== System Info ===================
AutoGluon Version: 1.2b20241127
Python Version: 3.11.9
Operating System: Linux
Platform Machine: x86_64
Platform Version: #1 SMP Tue Sep 24 10:00:37 UTC 2024
CPU Count: 8
Pytorch Version: 2.5.1+cu124
CUDA Version: 12.4
Memory Avail: 28.39 GB / 30.95 GB (91.7%)
Disk Space Avail: 187.71 GB / 255.99 GB (73.3%)
===================================================
AutoGluon infers your prediction problem is: 'binary' (because only two unique label-values observed).
2 unique label values: [1, 0]
If 'binary' is not the correct problem_type, please manually specify the problem_type parameter during Predictor init (You may specify problem_type as one of: ['binary', 'multiclass', 'regression', 'quantile'])
AutoMM starts to create your model. ✨✨✨
To track the learning progress, you can open a terminal and launch Tensorboard:
```shell
# Assume you have installed tensorboard
tensorboard --logdir /home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/tmp/4be46220e5714149aa509655e5fcc42b-automm_sst
```
Seed set to 0
GPU Count: 1
GPU Count to be Used: 1
GPU 0 Name: Tesla T4
GPU 0 Memory: 0.43GB/15.0GB (Used/Total)
Using 16bit Automatic Mixed Precision (AMP)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
| Name | Type | Params | Mode
---------------------------------------------------------------------------
0 | model | HFAutoModelForTextPrediction | 108 M | train
1 | validation_metric | MulticlassAccuracy | 0 | train
2 | loss_func | CrossEntropyLoss | 0 | train
---------------------------------------------------------------------------
108 M Trainable params
0 Non-trainable params
108 M Total params
435.573 Total estimated model params size (MB)
4 Modules in train mode
225 Modules in eval mode
Epoch 0, global step 3: 'val_acc' reached 0.56000 (best 0.56000), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/tmp/4be46220e5714149aa509655e5fcc42b-automm_sst/epoch=0-step=3.ckpt' as top 3
Epoch 0, global step 7: 'val_acc' reached 0.62500 (best 0.62500), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/tmp/4be46220e5714149aa509655e5fcc42b-automm_sst/epoch=0-step=7.ckpt' as top 3
Epoch 1, global step 10: 'val_acc' reached 0.70500 (best 0.70500), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/tmp/4be46220e5714149aa509655e5fcc42b-automm_sst/epoch=1-step=10.ckpt' as top 3
Epoch 1, global step 14: 'val_acc' reached 0.84500 (best 0.84500), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/tmp/4be46220e5714149aa509655e5fcc42b-automm_sst/epoch=1-step=14.ckpt' as top 3
Epoch 2, global step 17: 'val_acc' reached 0.89000 (best 0.89000), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/tmp/4be46220e5714149aa509655e5fcc42b-automm_sst/epoch=2-step=17.ckpt' as top 3
Epoch 2, global step 21: 'val_acc' reached 0.82500 (best 0.89000), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/tmp/4be46220e5714149aa509655e5fcc42b-automm_sst/epoch=2-step=21.ckpt' as top 3
Time limit reached. Elapsed time is 0:01:01. Signaling Trainer to stop.
Start to fuse 3 checkpoints via the greedy soup algorithm.
/home/ci/autogluon/multimodal/src/autogluon/multimodal/learners/base.py:2117: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
state_dict = torch.load(path, map_location=torch.device("cpu"))["state_dict"] # nosec B614
/home/ci/autogluon/multimodal/src/autogluon/multimodal/utils/checkpoint.py:45: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
state_dict = torch.load(per_path, map_location=torch.device("cpu"))["state_dict"] # nosec B614
/home/ci/autogluon/multimodal/src/autogluon/multimodal/utils/checkpoint.py:45: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
state_dict = torch.load(per_path, map_location=torch.device("cpu"))["state_dict"] # nosec B614
/home/ci/autogluon/multimodal/src/autogluon/multimodal/utils/checkpoint.py:63: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
avg_state_dict = torch.load(checkpoint_paths[0], map_location=torch.device("cpu"))["state_dict"] # nosec B614
AutoMM has created your model. 🎉🎉🎉
To load the model, use the code below:
```python
from autogluon.multimodal import MultiModalPredictor
predictor = MultiModalPredictor.load("/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/tmp/4be46220e5714149aa509655e5fcc42b-automm_sst")
```
If you are not satisfied with the model, try to increase the training time,
adjust the hyperparameters (https://auto.gluon.ai/stable/tutorials/multimodal/advanced_topics/customization.html),
or post issues on GitHub (https://github.com/autogluon/autogluon/issues).
<autogluon.multimodal.predictor.MultiModalPredictor at 0x7fce509c8f50>
训练后，我们可以在与训练数据格式相似的单独测试数据上评估我们的预测器：
test_score = predictor.evaluate(test_data)
print(test_score)
{'acc': 0.8864678899082569}
如果训练成功完成，可以在model_path下找到model.ckpt。如果你认为模型仍然欠拟合，可以通过再次运行.fit()来从这个检查点继续训练，使用相同的数据。如果你有一些新数据要添加并且不想从头开始训练，你也可以使用新的组合数据集运行.fit()。
predictor_2 = MultiModalPredictor.load(model_path) # you can also use the `predictor` we assigned above
train_data_2 = train_data.drop(train_data_1.index).sample(n=subsample_size, random_state=0)
predictor_2.fit(train_data_2, time_limit=60)
Load pretrained checkpoint: /home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/tmp/4be46220e5714149aa509655e5fcc42b-automm_sst/model.ckpt
/home/ci/autogluon/multimodal/src/autogluon/multimodal/learners/base.py:2117: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
state_dict = torch.load(path, map_location=torch.device("cpu"))["state_dict"] # nosec B614
A new predictor save path is created. This is to prevent you to overwrite previous predictor saved here. You could check current save path at predictor._save_path. If you still want to use this path, set resume=True
No path specified. Models will be saved in: "AutogluonModels/ag-20241127_095409"
=================== System Info ===================
AutoGluon Version: 1.2b20241127
Python Version: 3.11.9
Operating System: Linux
Platform Machine: x86_64
Platform Version: #1 SMP Tue Sep 24 10:00:37 UTC 2024
CPU Count: 8
Pytorch Version: 2.5.1+cu124
CUDA Version: 12.4
Memory Avail: 25.58 GB / 30.95 GB (82.7%)
Disk Space Avail: 186.92 GB / 255.99 GB (73.0%)
===================================================
AutoMM starts to create your model. ✨✨✨
To track the learning progress, you can open a terminal and launch Tensorboard:
```shell
# Assume you have installed tensorboard
tensorboard --logdir /home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/AutogluonModels/ag-20241127_095409
```
Seed set to 0
GPU Count: 1
GPU Count to be Used: 1
GPU 0 Name: Tesla T4
GPU 0 Memory: 0.58GB/15.0GB (Used/Total)
Using 16bit Automatic Mixed Precision (AMP)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
| Name | Type | Params | Mode
---------------------------------------------------------------------------
0 | model | HFAutoModelForTextPrediction | 108 M | train
1 | validation_metric | MulticlassAccuracy | 0 | train
2 | loss_func | CrossEntropyLoss | 0 | train
---------------------------------------------------------------------------
108 M Trainable params
0 Non-trainable params
108 M Total params
435.573 Total estimated model params size (MB)
229 Modules in train mode
0 Modules in eval mode
Epoch 0, global step 3: 'val_acc' reached 0.86500 (best 0.86500), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/AutogluonModels/ag-20241127_095409/epoch=0-step=3.ckpt' as top 3
Epoch 0, global step 7: 'val_acc' reached 0.83000 (best 0.86500), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/AutogluonModels/ag-20241127_095409/epoch=0-step=7.ckpt' as top 3
Epoch 1, global step 10: 'val_acc' reached 0.86500 (best 0.86500), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/AutogluonModels/ag-20241127_095409/epoch=1-step=10.ckpt' as top 3
Epoch 1, global step 14: 'val_acc' reached 0.88500 (best 0.88500), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/AutogluonModels/ag-20241127_095409/epoch=1-step=14.ckpt' as top 3
Epoch 2, global step 17: 'val_acc' reached 0.88500 (best 0.88500), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/AutogluonModels/ag-20241127_095409/epoch=2-step=17.ckpt' as top 3
Epoch 2, global step 21: 'val_acc' reached 0.90500 (best 0.90500), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/AutogluonModels/ag-20241127_095409/epoch=2-step=21.ckpt' as top 3
Time limit reached. Elapsed time is 0:01:02. Signaling Trainer to stop.
Start to fuse 3 checkpoints via the greedy soup algorithm.
/home/ci/autogluon/multimodal/src/autogluon/multimodal/learners/base.py:2117: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
state_dict = torch.load(path, map_location=torch.device("cpu"))["state_dict"] # nosec B614
/home/ci/autogluon/multimodal/src/autogluon/multimodal/utils/checkpoint.py:45: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
state_dict = torch.load(per_path, map_location=torch.device("cpu"))["state_dict"] # nosec B614
/home/ci/autogluon/multimodal/src/autogluon/multimodal/utils/checkpoint.py:45: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
state_dict = torch.load(per_path, map_location=torch.device("cpu"))["state_dict"] # nosec B614
/home/ci/autogluon/multimodal/src/autogluon/multimodal/utils/checkpoint.py:63: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
avg_state_dict = torch.load(checkpoint_paths[0], map_location=torch.device("cpu"))["state_dict"] # nosec B614
AutoMM has created your model. 🎉🎉🎉
To load the model, use the code below:
```python
from autogluon.multimodal import MultiModalPredictor
predictor = MultiModalPredictor.load("/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/AutogluonModels/ag-20241127_095409")
```
If you are not satisfied with the model, try to increase the training time,
adjust the hyperparameters (https://auto.gluon.ai/stable/tutorials/multimodal/advanced_topics/customization.html),
or post issues on GitHub (https://github.com/autogluon/autogluon/issues).
<autogluon.multimodal.predictor.MultiModalPredictor at 0x7fce4fec5490>
test_score_2 = predictor_2.evaluate(test_data)
print(test_score_2)
{'acc': 0.8887614678899083}
用例2：从最后一个检查点恢复训练¶
如果你的训练过程因某些原因崩溃，AutoMM 允许你从上次中断的地方恢复训练。last.ckpt 将被保存在 model_path 下，而不是 model.ckpt。要恢复训练，你只需调用 MultiModalPredictor.load() 并带上 resume 选项：
predictor_resume = MultiModalPredictor.load(path=model_path, resume=True)
predictor.fit(train_data, time_limit=60)
用例3：将预训练模型应用于新任务¶
通常，你会遇到这样的情况：一个新任务与你之前训练模型的任务相关但不完全相同（例如，训练一个更细粒度的情感分析模型，或者为你的多类模型添加更多类别）。如果你希望利用模型从旧数据中学到的知识，以帮助它更快更有效地学习新任务，AutoMM支持将你训练好的模型转储为模型权重，并将它们用作基础模型：
dump_model_path = f"./tmp/{uuid.uuid4().hex}-automm_sst"
predictor.dump_model(save_path=dump_model_path)
Model weights and tokenizer for hf_text are saved to ./tmp/b2d8f816bc8a423cb76370f1c439e040-automm_sst/hf_text.
'./tmp/b2d8f816bc8a423cb76370f1c439e040-automm_sst'
然后，您可以加载训练好的模型的权重，并在新数据上继续训练/微调模型。
这里有一个例子，使用了我们之前在回归任务上训练的二进制文本模型。我们使用语义文本相似性基准数据集仅用于说明，因此您可能希望将此功能应用于更相关的数据集。在这些数据中，名为score的列包含数值（我们想要预测的），这些数值是人工标注的每对句子的相似性分数。
sts_train_data = load_pd.load("https://autogluon-text.s3-accelerate.amazonaws.com/glue/sts/train.parquet")[
["sentence1", "sentence2", "score"]
]
sts_test_data = load_pd.load("https://autogluon-text.s3-accelerate.amazonaws.com/glue/sts/dev.parquet")[
["sentence1", "sentence2", "score"]
]
sts_train_data.head(10)
Loaded data from: https://autogluon-text.s3-accelerate.amazonaws.com/glue/sts/train.parquet | Columns = 4 / 4 | Rows = 5749 -> 5749
Loaded data from: https://autogluon-text.s3-accelerate.amazonaws.com/glue/sts/dev.parquet | Columns = 4 / 4 | Rows = 1500 -> 1500
| sentence1 | sentence2 | score | |
|---|---|---|---|
| 0 | A plane is taking off. | An air plane is taking off. | 5.00 |
| 1 | A man is playing a large flute. | A man is playing a flute. | 3.80 |
| 2 | A man is spreading shreded cheese on a pizza. | A man is spreading shredded cheese on an uncoo... | 3.80 |
| 3 | Three men are playing chess. | Two men are playing chess. | 2.60 |
| 4 | A man is playing the cello. | A man seated is playing the cello. | 4.25 |
| 5 | Some men are fighting. | Two men are fighting. | 4.25 |
| 6 | A man is smoking. | A man is skating. | 0.50 |
| 7 | The man is playing the piano. | The man is playing the guitar. | 1.60 |
| 8 | A man is playing on a guitar and singing. | A woman is playing an acoustic guitar and sing... | 2.20 |
| 9 | A person is throwing a cat on to the ceiling. | A person throws a cat on the ceiling. | 5.00 |
要指定您创建的自定义模型，请在.fit()中使用hyperparameters选项：
hyperparameters={
"model.hf_text.checkpoint_name": dump_model_path
}
sts_model_path = f"./tmp/{uuid.uuid4().hex}-automm_sts"
predictor_sts = MultiModalPredictor(label="score", path=sts_model_path)
predictor_sts.fit(
sts_train_data, hyperparameters={"model.hf_text.checkpoint_name": f"{dump_model_path}/hf_text"}, time_limit=30
)
=================== System Info ===================
AutoGluon Version: 1.2b20241127
Python Version: 3.11.9
Operating System: Linux
Platform Machine: x86_64
Platform Version: #1 SMP Tue Sep 24 10:00:37 UTC 2024
CPU Count: 8
Pytorch Version: 2.5.1+cu124
CUDA Version: 12.4
Memory Avail: 24.35 GB / 30.95 GB (78.7%)
Disk Space Avail: 186.11 GB / 255.99 GB (72.7%)
===================================================
AutoGluon infers your prediction problem is: 'regression' (because dtype of label-column == float and label-values can't be converted to int).
Label info (max, min, mean, stddev): (5.0, 0.0, 2.701, 1.4644)
If 'regression' is not the correct problem_type, please manually specify the problem_type parameter during Predictor init (You may specify problem_type as one of: ['binary', 'multiclass', 'regression', 'quantile'])
AutoMM starts to create your model. ✨✨✨
To track the learning progress, you can open a terminal and launch Tensorboard:
```shell
# Assume you have installed tensorboard
tensorboard --logdir /home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/tmp/5c60190e7e214a019ce44beb177e5de0-automm_sts
```
Seed set to 0
GPU Count: 1
GPU Count to be Used: 1
GPU 0 Name: Tesla T4
GPU 0 Memory: 0.58GB/15.0GB (Used/Total)
Using 16bit Automatic Mixed Precision (AMP)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
| Name | Type | Params | Mode
---------------------------------------------------------------------------
0 | model | HFAutoModelForTextPrediction | 108 M | train
1 | validation_metric | MeanSquaredError | 0 | train
2 | loss_func | MSELoss | 0 | train
---------------------------------------------------------------------------
108 M Trainable params
0 Non-trainable params
108 M Total params
435.570 Total estimated model params size (MB)
4 Modules in train mode
225 Modules in eval mode
Epoch 0, global step 20: 'val_rmse' reached 0.59089 (best 0.59089), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/tmp/5c60190e7e214a019ce44beb177e5de0-automm_sts/epoch=0-step=20.ckpt' as top 3
Time limit reached. Elapsed time is 0:00:30. Signaling Trainer to stop.
Epoch 0, global step 35: 'val_rmse' reached 0.53419 (best 0.53419), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/tmp/5c60190e7e214a019ce44beb177e5de0-automm_sts/epoch=0-step=35.ckpt' as top 3
Start to fuse 2 checkpoints via the greedy soup algorithm.
/home/ci/autogluon/multimodal/src/autogluon/multimodal/learners/base.py:2117: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
state_dict = torch.load(path, map_location=torch.device("cpu"))["state_dict"] # nosec B614
/home/ci/autogluon/multimodal/src/autogluon/multimodal/utils/checkpoint.py:45: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
state_dict = torch.load(per_path, map_location=torch.device("cpu"))["state_dict"] # nosec B614
/home/ci/autogluon/multimodal/src/autogluon/multimodal/utils/checkpoint.py:63: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
avg_state_dict = torch.load(checkpoint_paths[0], map_location=torch.device("cpu"))["state_dict"] # nosec B614
AutoMM has created your model. 🎉🎉🎉
To load the model, use the code below:
```python
from autogluon.multimodal import MultiModalPredictor
predictor = MultiModalPredictor.load("/home/ci/autogluon/docs/tutorials/multimodal/advanced_topics/tmp/5c60190e7e214a019ce44beb177e5de0-automm_sts")
```
If you are not satisfied with the model, try to increase the training time,
adjust the hyperparameters (https://auto.gluon.ai/stable/tutorials/multimodal/advanced_topics/customization.html),
or post issues on GitHub (https://github.com/autogluon/autogluon/issues).
<autogluon.multimodal.predictor.MultiModalPredictor at 0x7fce88df5f90>
test_score = predictor_sts.evaluate(sts_test_data, metrics=["rmse", "pearsonr", "spearmanr"])
print("RMSE = {:.2f}".format(test_score["rmse"]))
print("PEARSONR = {:.4f}".format(test_score["pearsonr"]))
print("SPEARMANR = {:.4f}".format(test_score["spearmanr"]))
RMSE = 0.79
PEARSONR = 0.8632
SPEARMANR = 0.8645
我们目前支持导出timm图像模型、MMDetection图像模型、HuggingFace文本模型以及包含上述模型的任何融合模型。同样，我们也可以加载一个自定义训练的timm图像模型：
{"model.timm_image.checkpoint_name": timm_image_model_path}
以及一个自定义训练的 MMDetection 模型，具有以下特点：
{"model.mmdet_image.checkpoint_name": mmdet_image_model_path}
此功能帮助您将之前训练任务的知识应用到新任务上，从而节省您的时间和计算资源。我们不会在本教程中深入细节，但请记住，我们尚未解决此用例中的一个重大挑战，即灾难性遗忘。