稀疏化

高级API，用于使用各种算法自动稀疏化您的模型。

函数

sparsify	稀疏化给定模型并搜索最优稀疏化权重。
export	将稀疏动态模型导出为常规模型。

export(model)

将稀疏动态模型导出为常规模型。

这应该在模型微调并且权重固定之后完成。

警告

在调用export()之后，稀疏掩码将不再强制执行。这意味着任何未来的权重更新都会破坏稀疏模式。如果你想继续训练，请在训练结束后调用export()。

Parameters:

模型 (模块) –

Return type:

模块

sparsify(model, mode, config=None)

稀疏化给定模型并搜索最优稀疏化权重。

Parameters:

• model (Module) – 一个标准模型，包含可以就地稀疏化的标准构建块。

• mode (_ModeDescriptor | str | List[_ModeDescriptor | str] | List[Tuple[str, Dict[str, Any]]]) –

  A (list of) string(s) or Mode(s) or a list of tuples containing the mode and its config indicating the desired mode(s) (and configurations) for the convert process. Modes set up the model for different algorithms for model optimization. The following modes are available:

  • "sparse_magnitude": The model will be sparsified according to the magnitude of weights in each layer. The mode’s config is described in SparseMagnitudeConfig.

  • "sparsegpt": The model will be sparsified and weights are updated optimally using an Hessian approximation of the loss function (see SparseGPT paper for details). The mode’s config is described in SparseGPTConfig.

  If the mode argument is specified as a dictionary, the keys should indicate the mode and the values specify the per-mode configuration. If not provided, then default configuration would be used.

• config (Dict[str, Any] | None) –

  Additional optional arguments to configure the search. Currently, we support:

  • verbose: Whether to print detailed search stats during search.

  • forward_loop: A Callable that takes a model as input and runs a forward loop

    on it. It is recommended to choose the data loader used inside the forward loop carefully to reduce the runtime. Cannot be provided at the same time as data_loader and collect_func.

  • data_loader: An iterator yielding batches of data for calibrating the normalization layers in the model or compute gradient scores. It is recommended to use the same data loader as for training but with significantly fewer iterations. Cannot be provided at the same time as forward_loop.

  • collect_func: A Callable that takes a batch of data from the data loader as input and returns the input to model.forward() as described in run_forward_loop. Cannot be provided at the same time as forward_loop.

  Note

  Additional configuration options may be added by individual algorithms. Please refer to the documentation of the individual algorithms for more information.

Return type:

元组[模块, 字典[字符串, 任意类型]]

返回：一个稀疏化的模型

注意

给定的模型是就地稀疏化的。因此，返回的模型是对输入模型的同一模型实例的引用。