The board of traffic light is located in input image by its color and shape. The color image of the board is converted to YCbCr space. The region that includes main color(red, amber, green) is segmented by thresholds. The traffic light is positioned by morphological properties. The feature of region of interest(RoI) is extracted by Gabor wavelet and 2 dimension independent component analysis, and sent into the nearest neighbor classifier to classify the type of traffic light. Structures of hidden Markov model are built by several representative observation sequences. The current state of traffic light is estimated by hidden Markov model which combines with recognition and tracking results. The experimental results show that this algorithm can recognize and track the traffic lights reliably and accurately, and estimate the states of the traffic light effectively.