Low-Rank Adaptation (LoRA) - Page 5

Low-rank adaptation, known as LoRA, fine-tunes a model by training small added matrices while leaving the original weights frozen. The method assumes that useful task-specific changes can be represented with far fewer parameters than the full model contains. This lowers memory requirements, shortens training, and makes it practical to store separate adapters for different domains or styles. LoRA is common with large language and image models, although adapter quality still depends on data, rank, target layers, and evaluation. It does not update a model’s knowledge automatically or remove inherited risks, and combining several adapters can create interactions that require additional testing before deployment.