基于自监督对抗学习的多尺度知识蒸馏方法
CSTR:
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

TP391

基金项目:

安徽理工大学医学专项培育项目(YZ2023H2C005);国家自然科学基金项目(62102003);安徽理工大学环境友好材料与职业健康研究院研发专项基金项目(ALW2021YF04);安徽理工大学研究生创新基金项目(2023cx2139).


Multi-scale knowledge distillation method based on self-supervised adversarial learning
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    针对离线知识蒸馏中因教师与学生之间规模差距过大, 知识难以有效传递,导致学生性能不佳的问题, 提出一种基于自监督对抗学习的多尺度知识蒸馏方法(SAMKD), 并利用自监督和对抗学习进一步开发中间多尺度特征与网络末端输出特征 logits 的潜力. 首先, 引入多角度几何变换图像监督网络学习; 然后, 设计多分支辅助网络提取主干网络的多尺度特征, 进而获得更多监督信息; 最后, 利用对抗学习的二元博弈思想进行多阶段对抗训练, 将多层次的知识通过蒸馏方法充分传递. 在3个具有挑战性的公开数据集 CIFAR-10、CIFAR-100和Tiny-ImageNet上进行广泛评估, 实验结果表明所提出方法相较其他先进知识蒸馏方法具有强大的竞争力.

    Abstract:

    To address the challenge of ineffective knowledge transfer and poor student performance in offline knowledge distillation due to the significant scale gap between teachers and students, a multi-scale knowledge distillation method based on self-supervised adversarial learning (SAMKD) is proposed. This method leverages self-supervision and adversarial learning to further develop the potential of intermediate multi-scale features and network logits. Firstly, the paper introduces supervised network learning using multi-angle geometrically transformed images. Then, it designs a multi-branch auxiliary network to extract multi-scale features from the backbone network, thereby enhancing supervisory information. Finally, it employs a binary adversarial training approach inspired by adversarial learning for multi-stage adversarial training, effectively facilitating comprehensive knowledge transfer across multiple levels through distillation. Extensive evaluations on three challenging public datasets, CIFAR-10, CIFAR-100, and Tiny-ImageNet, demonstrate that the proposed method exhibits robust competitiveness and outperforms other state-of-the-art knowledge distillation methods.

    参考文献
    相似文献
    引证文献
引用本文

张建,梁兴柱,张康,等.基于自监督对抗学习的多尺度知识蒸馏方法[J].控制与决策,2025,40(3):880-888

复制
相关视频

分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-03-20
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2025-02-11
  • 出版日期: 2025-03-20
文章二维码