基于通道性能度量的神经网络结构搜索算法
CSTR:
作者:
作者单位:

1. 中国矿业大学 信息与控制工程学院,江苏 徐州 221116;2. 中国矿业大学 机电工程学院,江苏 徐州 221116

作者简介:

通讯作者:

E-mail: panjie1616@cumt.edu.cn.

中图分类号:

TP183

基金项目:

国家自然科学基金项目(62176258,62273349,52174152);中央高校基本科研业务费专项资金项目(2021YCPY0111);江苏省高校优势学科建设工程资助项目(PAPD).


Differentiable neural architecture search with channel performance measurement
Author:
Affiliation:

1. School of Information and Control Engineering,China University of Mining and Technology,Xuzhou 221116,China;2. School of Mechatronic Engineering,China University of Mining and Technology,Xuzhou 221116,China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    卷积神经网络的表征与预测能力往往依赖结构合理性,但其主流结构均由人工设计,存在设计难度高、算力要求强、时间开销大等问题.如何让神经网络自主搜索合理结构并节约计算资源是当前的研究重点.目前,基于部分通道连接的可微分结构搜索算法,以其高效的显存利用率在搜索速度和分类性能上表现良好.然而,其针对通道的随机采样策略易造成重要信息丢失,当通道连接不足时性能明显下降.为此,提出一种基于通道性能度量的神经网络结构搜索算法,利用注意力机制提取通道重要性系数,并以此对通道进行排序采样.此外,考虑到预热阶段导致搜索不充分,产生较大离散化误差,在结构权重连续化的过程中设计温度正则化系数,提升权重差异.实验表明,所提算法能够在节约计算资源的基础上搜索出更优的卷积神经网络结构.

    Abstract:

    The design of convolutional neural network architecture is critical for achieving high characterization and prediction performance . However, manual design can be time-consuming and computationally expensive. Automatic architecture search methods are thus becoming popular. Presently, partial channel connections for memory-efficient differentiable architecture search being efficient in terms of search speed, memory utilization, and classification performance. However, the random sampling strategy can cause the loss of important information. To address these issues, this paper proposes differentiable neural architecture search with channel performance measurement, which extracts importance coefficients of channels with attention mechanism and ranks the channels accordingly to select the most important ones. The proposed algorithm also includes a temperature regularization coefficient to improve the transformation from discrete to continuous architecture weight and reduce discretization errors. Experimental results show that the proposed algorithm achieves good performance while significantly reducing computational resources.

    参考文献
    相似文献
    引证文献
引用本文

潘杰,郑学驰,邹筱瑜.基于通道性能度量的神经网络结构搜索算法[J].控制与决策,2024,39(7):2151-2160

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2024-06-06
  • 出版日期: 2024-07-20
文章二维码