From full fine-tuning to parameter-efficient adaptation: a taxonomy of deep transfer learning strategies

Wenkai Zhang

doi:10.47297/taposatWSP2633-456923.20250605

PDF
Export
Share
Collection
Album

From full fine-tuning to parameter-efficient adaptation: a taxonomy of deep transfer learning strategies

Volume 6, Issue 5

Views: 0 Downloads: 0

From full fine-tuning to parameter-efficient adaptation: a taxonomy of deep transfer learning strategies
From full fine-tuning to parameter-efficient adaptation: a taxonomy of deep transfer learning strategies
Electronic Information Science and Technology, Nanjing Agricultural University, Nanjing, Jiangsu, 211800, China
Vol. 6, Issue 5, Pages: 118-122(2025)
Published：19 May 2025
DOI：10.47297/taposatWSP2633-456923.20250605
Accepted：

Scan QR Code

Zhang Wenkai.From full fine-tuning to parameter-efficient adaptation: a taxonomy of deep transfer learning strategies[J].Theory and Practice of Science and Technology,2025,06(05):118-122.
DOI：

Zhang Wenkai.From full fine-tuning to parameter-efficient adaptation: a taxonomy of deep transfer learning strategies[J].Theory and Practice of Science and Technology,2025,06(05):118-122. DOI： 10.47297/taposatWSP2633-456923.20250605.

摘要

Abstract

This paper focuses on the field of deep transfer learning

aiming to study the dual dilemma of traditional full-parameter fine-tuning in terms computing power demand and overfitting risk. By establishing a three-level taxonomy of parameter-efficient transfer learning

this paper analyzes in detail the methods such as parameter addition strategy parameter reparameterization

and parameter subset selection. It also refines the general principles such as minimal disturbance of feature space

task-knowledge decoupling

and incremental knowledge injection Simultaneously

it points out the current challenges of lack of theoretical interpretability (black-box nature) and weak cross-architecture generalization ability

and proposes future directions such as-explanation-based interpretable adaption and neural architecture search (NAS)-driven automatic adaption

aiming to construct a more systematic and efficient theoretical framework for deep transfer learning.

关键词

Keywords

references

Wang Yitong. Research on Cross-domain Sentiment Classification Based on Pretraining and Adversarial Learning [D]. Beijing University of Posts and Telecommunications, 2023.

Wu Peng. Research on Fine-grained Bird Classification Model Based on Transfer Learning [D]. Beijing Forestry University, 2021.

Wang Keli, Yuan Hongchun. Image Recognition Method of Aquatic Animals Based on Transfer Learning [J]. Journal of Computer Applications, 2018, 38(5): 6. DOI: 10.11772/j.issn.1001-9081.2017102487.

Zang Haixiang, Guo Jingwei, Huang Manyun, et al. State Estimation of Power Systems with Time-varying Topology Based on Deep Transfer Learning [J]. Automation of Electric Power Systems, 2021, 45(24): 8.

Wang Pengde, Zhang Jingzhou. Research on Text Sentiment Analysis Based on MMD Dual-module Parameter Transfer [J]. Information Technology and Informatization, 2022(4): 65-68.

Zheng Zongsheng, Hu Chenyu, Huang Dongmei, et al. Research on Typhoon Grade Classification Based on Transfer Learning and Meteorological Satellite Cloud Images [J]. Remote Sensing Technology and Application, 2020, 35(1): 9. DOI: CNKI:SUN:YGJS.0.2020-01-020.

Zhou Qing, Zhou Jieqiong, Huang Yuan. Intelligent Project-based Learning: Immerse Every Child in "Real Learning"—Thoughts on the Project-based Teaching Case of "Making a Simple Intelligent Body Temperature Monitoring and Early Warning System" [J]. Inside and Outside the Classroom (Junior High School Edition), 2024(44): 1-3.

Wang Huimei, He Jiawei, Liu Jian, et al. Small Sample DDoS Attack Detection Method Based on Deep Transfer Learning. CN202010943146.5 [Retrieval Date: 2025-05-09].

Huang Yijing, Lü Junyan, Li Meng, et al. Auxiliary Diagnosis Algorithm for Diabetic Retinopathy Based on Transfer Learning [J]. Chinese Journal of Experimental Ophthalmology, 2019, 37(8): 5. DOI: 10.3760/cma.j.issn.2095-0160.2019.08.003.

Wang Pin, Li Linyu, Li Yongming, et al. Deep Transfer Semi-supervised Domain Adaptation Classification Method for Histopathological Images: CN202110096888.3 [P]. CN112733859A [Retrieval Date: 2025-05-09].

Li Yuchen. Research on Motor Imagery EEG Classification Based on Convolutional Neural Network [D]. Xidian University, 2020.

Dai Zuhua, Mou Qiaoling, Li Hongyi, et al. Deep Transfer Learning Method for Text Sentiment Classification: CN202010550138.4 [P]. CN111680160A [Retrieval Date: 2025-05-09].

Meng Zeqi. Developing Students' Cognitive Structure Based on Discovery Learning—Taking "Plane Rectangular Coordinate System" as an Example [J]. Innovation and Practice of Teaching Methods, 2020, 3(1): 72. DOI: 10.26549/jxffcxysj.v3i1.3150.

Li Tong. Research on Teaching Strategies of Information Technology Classroom to Promote Deep Learning [J]. Advances in Education, 2025, 15(3): 7. DOI: 10.12677/ae.2025.153388.

Dong Peihao, Jia Jibin, Zhou Fuhui, et al. Broadband Spectrum Sensing of Low-altitude UAV Swarms Based on Differential Privacy Federated Learning [J]. Journal of Electronics and Information Technology, 2025, 47(5): 1-11. DOI: 10.11999/JEIT241042.

Wu Tianqi, Zhao Ruirui, Zuo Anxin, et al. Velocity Picking Based on Multi-feature Fusion Network [C]//Proceedings of the 3rd China Petroleum Geophysical Exploration Academic Annual Conference (IV). 2025.

Yang Hailong, Yuan Yiping, Fan Panpan, et al. Foreign Object Recognition of Iron Removers for Mine Belt Conveyors Based on Transfer Learning and EfficientNet [J]. Coal Science and Technology, 2025, 53: 1-11. DOI: 10.12438/cst.2024-0772.

Wang Jiatian, Li Fanchang. Research on a Linear Transfer Meta-learning Algorithm [J]. Journal of Computer Engineering & Applications, 2025, 61(5). DOI: 10.3778/j.issn.1002-8331.2310-0321.

Feng Yunwen, Pan Weihuang, Lu Cheng, et al. Hydraulic Condition Monitoring and Fault Diagnosis of Civil Aircraft Based on Fault Logic [J]. Systems Engineering and Electronics, 2025, 47(3): 842-854. DOI: 10.12305/j.issn.1001-506X.2025.03.16.

Wang Jing. Research on Text Representation and Classification Based on Deep Learning [D]. Beijing University of Posts and Telecommunications, 2019.

Alert me when the article has been cited

Submit

No data

Related Author

No data

Related Institution

No data

⁰