Helper methods
A collection of helper functions for PEFT.
Checking if a model is a PEFT model
peft.helpers.check_if_peft_model
< source >( model_name_or_path: str ) → bool
Check if the model is a PEFT model.
Temporarily Rescaling Adapter Scale in LoraLayer Modules
peft.helpers.rescale_adapter_scale
< source >( model multiplier )
Context manager to temporarily rescale the scaling of the LoRA adapter in a model.
The original scaling values are restored when the context manager exits. This context manager works with the transformers and diffusers models that have directly loaded LoRA adapters.
For LoRA, applying this context manager with multiplier in [0, 1] is strictly equivalent to applying wise-ft (see #1940 for details). It can improve the performances of the model if there is a distribution shiftbetween the training data used for fine-tuning, and the test data used during inference.
Warning: It has been reported that when using Apple’s MPS backend for PyTorch, it is necessary to add a short sleep time after exiting the context before the scales are fully restored.