In order to realize the intelligent interconnection of human-machine-things as well as the efficient interaction between intelligent agents, the sixth generation of mobile communications system(6G) needs to make full use of the potential sensing function of the communication device to integrate communication and sensing. In this way, the promised performance requirements can be satisfied, including extremely high data rate, ultra-low latency, high-precision-wide-range sensing, etc. This survey summarizes the state of the arts from the perspectives of information theory, and then the critical challenges, such as integration waveform techniques, multiple access of integrated communication and sensing, and interference suppressions, are discussed. In particular, the research fruits of integrated communication and sensing in the Terahertz frequency band are exhibited in detail. At last, several open issues are posted at the end of the survey to indicate the research directions.