基于深度稀疏低秩分解的深度神经网络轻量化方法
作者:
作者单位:

四川九洲电器集团有限责任公司 四川绵阳 621000

作者简介:

通讯作者:

中图分类号:

TP391.4

基金项目:


Lightweight Method of Deep Neural Network Based on Deep Sparse Low Rank Decomposition
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    基于嵌入式平台对深度神经网络轻量化的需求,结合模块化、逐层处理思想,以主流检测识别深度神经网络Faster RCNN轻量化为目标,设计了基于深度稀疏低秩分解的轻量化方法。针对Faster RCNN网络架构特点,首先,采用深度可分离卷积和稀疏低秩理论对Faster RCNN网络的特征提取主干网络部分进行初始轻量化;其次,采用稀疏低秩裁剪对主干网络进行“逐层通道裁剪,逐层重训练,逐层调优”轻量化;然后,采用张量Tensor-Train分解理论,对区域建议网络进行轻量化处理,尽可能保证低性能损失;再次对识别与分类网络也进行稀疏低秩分解和通道裁剪,增加模型压缩倍数,减少所需要和所消耗计算资源;最后,基于感兴趣区域定位感知的RPN网络输入特征知识蒸馏,提升检测识别性能。数值实验表明,该方法可以实现模型压缩100倍,检测识别率仅下降5%。

    Abstract:

    Based on the requirement of embedded devices for deep neural network lightweight, and combined with the idea of modularization and layer by layer processing, a lightweight method based on deep sparse low rank decomposition is designed aiming at the lightweight of the mainstream detection and recognition network Faster RCNN. In view of characteristics of the Faster RCNN network architecture, firstly, initially lighten the backbone part of the Faster RCNN feature extraction network is realized through deep separable convolution and sparse low-rank theory; Secondly, sparse low-rank pruning is used to further lighten the backbone network in the way of “layer by layer channel pruning, layer by layer retraining, and layer by layer tuning”; Then, the region proposal network is lightened based on the Tensor-train decomposition theory, and the performance loss is ensured as low as possible; Sparse low rank decomposition and channel pruning are applied to the recognition and classification network again, which results in more compression times, less memory and less computing resources required; Finally, the input feature knowledge distillation of RPN network based on region of interest location perception improves the detection and recognition performance. Numerical experiments show that this method can achieve model compression by 100 times, and the detection and recognition rate is only reduced by 5%

    参考文献
    相似文献
    引证文献
引用本文
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2021-07-13
  • 最后修改日期:2021-12-04
  • 录用日期:2021-12-09
  • 在线发布日期: 2022-01-02
  • 出版日期: