基于注意力的共享参数胶囊网络
DOI:
作者:
作者单位:

上海理工大学

作者简介:

通讯作者:

中图分类号:

TP273

基金项目:

国家自然科学基金项目(面上项目,重点项目,重大项目),


Attention-based capsule network with shared parameters
Author:
Affiliation:

University of Shanghai for Science and Technology

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    针对传统胶囊网络特征信息的传播冗余性和解构低效性问题,本文提出了一种共享参数的注意力胶囊网络。该网络的优点主要体现在以下两个方面:1)提出了注意力机制的动态路由方法。通过计算低级胶囊的相关性,使得在保留特征空间信息的同时更加关注相关性高的特征信息,并完成前向传播;2)在动态路由层提出了共享转换矩阵。基于低级胶囊投票一致性对高级胶囊激活,并通过共享转换矩阵减少模型的参数量同时实现改进胶囊网络的稳健性。五个公开数据集的分类对比实验结果表明,本文提出的胶囊网络在Fashion-MNIST、SVHN和CIFAR10数据集上分别取得5.17%、3.67%和9.35%的最好分类结果,还在复杂数据集上具有显著的白盒对抗攻击鲁棒性。此外,在基于smallNORB和affNISH公开数据集的仿射变换对比实验表明,本文提出的胶囊网络具有显著的仿射变换鲁棒性。最后,计算效率分析对比实验结果表明,本文提出的共享参数胶囊网络在不增加浮点运算的情况下,参数量比传统的胶囊网络减少了4.9%,具有突出的计算量优势。

    Abstract:

    Aiming to handle the problem of propagation redundancy and deconstruction inefficiency of features in traditional capsule networks, this paper proposes an attention-based capsule network with shared parameters. The merits of such a network lie mainly in the following two issues: 1) a dynamic routing method based on an attention mechanism is proposed. This method calculates the correlation between low-level capsules to maintain the space information of features and pay more attention to the feature information with a high correlation, thus fulfilling the forward propagation; 2) a shared transformation matrix is developed in the dynamic routing layer. The high-level capsules are activated based on the voting consistency of the low-level capsules. Then, the transformation matrix with shared parameters is used to reduce the parameters of the model and obtain the robustness of the capsule network. Experimental results of comparison classification on five public datasets show that the capsule network proposed in this paper achieves the best classification results of 5.17%, 3.67% and 9.35% on the Fashion-MNIST, SVHN and CIFAR10 datasets, respectively. Moreover, it has significant robustness against the white-box anti-attack. In addition, the transformation experimental results on smallNORB and affNISH public datasets show that the proposed capsule network has obvious robustness to the transformation. Finally, the experimental results of computational efficiency show that the proposed capsule network with shared parameters reduces the parameters of traditional capsule networks by 4.9% without adding floating-point operations and has an overwhelming advantage in computation.

    参考文献
    相似文献
    引证文献
引用本文
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2021-10-25
  • 最后修改日期:2022-03-08
  • 录用日期:2022-03-15
  • 在线发布日期:
  • 出版日期: