DOI:

tp18

Overview of Recurrent Neural Networks
Author:
Affiliation:

China university of Petroleum(Beijing)

Fund Project:

the Science Foundation of China University of Petroleum, Beijing (No.2462020YXZZ023)

• 摘要
• |
• 图/表
• |
• 访问统计
• |
• 参考文献
• |
• 相似文献
• |
• 引证文献
• |
• 资源附件
• |
• 文章评论
摘要:

循环神经网络是神经网络序列模型的主要实现形式，并在最近几年得到了迅速发展。循环神经网络现在基本是机器翻译，机器问题回答，序列视频分析的标准处理手段，也是对于手写体自动合成，语音处理和图像生成等问题的主流建模手段。本文对循环神经网络的各分支按照网络结构进行详细分类，大致分为三个大类：第一类是衍生循环神经网络，这一类网络是基于基本RNNs模型的结构衍生变体，即对RNNs的内部结构进行修改，包括双向循环神经网络、长短期记忆网络、微分循环神经网络、高速公路网、多维循环神经网络和嵌套堆叠循环神经网络；第二类是组合循环神经网络，这一类网络是将一些经典的其他网络模型或结构和第一类衍生循环神经网络进行组合，得到更好的模型效果，是一种非常有效的手段，包括卷积循环神经网络、网格循环神经网络、图循环神经网络、暂态循环神经网络、格子循环神经网络、分层循环神经网络和记忆循环神经网络；第三类是混合循环神经网络，这一类网络模型既有不同网络模型的组合，又在RNNs内部结构上进行了修改，是同属于前两类网络分类的结构。为了对循环神经网络理解的更加深入，本文还将介绍和循环神经网络经常被混为一谈的递归神经网络结构以及递归神经网络与循环神经网络的区别与联系。在详略描述上述模型的应用背景、网络结构以及模型变种后，对各个模型的特点进行总结和比较，并在最后对循环神经网络模型进行展望和总结。

Abstract:

Recurrent neural networks(RNNs) are the main implementation paradigms for deep neural network sequence model, and has been developed rapidly and widespreadly in last two decades. Now, RNNs are cornerstone and foundation underpinning for machine translation, machine question answering and sequence video analysis, and RNNs are also the mainstream modeling approaches for handwriting automatic synthesis, speech processing and image generation. In this paper, the branches of recurrent neural networks are classified in detail according to the network structure, which can be roughly divided into three categories: the first one is all sorts of variants of recurrent neural networks, which is a structural variants based on the basic RNNs architecture, that is, modifying the internal structure of RNNs, including bidirectional RNNs, long-term and short-term memory networks, differential RNNs, highway Network, multidimensional RNNs and nested stacked RNNs. The second kind is Combined RNNs, which combines some classical other network models or structures with the first kind of RNNs to get better modeling effect. It is a very effective means, including Convolution RNNs, Grid RNNs, Graph RNNs, Temporal RNNs, Lattice RNNs, Hierarchical RNNs and Memory RNNs. The third one is Hybrid RNNs, which not only combines different network models, but also modifies the internal structure of RNNs. In order to understand the RNNs more deeply, this paper will also introduce the structure of recursive neural networks which is often confused with RNNs, and the difference and connection between recursive neural networks and RNNs. After a detailed description of the application background, network structure and model variants of the above models based on RNNs, the characteristics of each model are summarized and compared. Finally, the prospect and summary of the RNNs are given.

参考文献
相似文献
引证文献

• 点击次数:
• 下载次数:
• HTML阅读次数:
• 引用次数:
##### 历史
• 收稿日期:2021-07-16
• 最后修改日期:2021-11-11
• 录用日期:2021-11-12
• 在线发布日期:
• 出版日期: