Handwritten Chinese text recognition which involves thousands of character categories, variant writing styles and monotonous data collection process is a long-standing focus in the field of pattern recognition research. In response to these issues, we propose a residual attention network of segmentation-free and recurrent-free for end-to-end handwritten text recognition with ResNet-26 as the main architecture, using depthwise separable convolution to extract the representation features. The residual attention gate block enhances the important of the key areas of input text image. Then, the text-lines up-sampling of batch bilinear interpolation is used to implement the mapping from two dimension text representation to one dimension text line representation. Finally, connectionist temporal classification as the loss function is employed to realize the corresponding relationship between the high-level extraction representation features and the character sequence labels. An experimental study is carried out on two datasets of CASIA-HWDB2.x and ICDAR2013, and the results indicate that the method can effectively implement end-to-end handwritten text recognition without any position information of characters or text lines, and superior to the existing research methods.