To forecast the demand of the taxi, it is significant part of smart city and intelligent traffic system to use large amount of off-line GPS data. A deep learning-based, CNN-LSTM-ResNet, is proposed for the demand of taxi in this paper. We converted GPS data of taxi and weather data into raster data, and put them into the model as input to obtain the predictions. Firstly, Convolutional Neural Network (CNN) is used to extract the spatial features of urban traffic flow, and Residual Units to deepen the layers of network, then to extract the temporal proximity, periodicity and tendency of the GPS data, Long Short-Term Memory (LSTM) is used. Finally, to predict the demand of taxi in specific areas of the city, three components are fused by the corresponding weights, and the syncretic result is combined with external factors, like the weather, holiday and air quality index. The experiments are conducted on taxi GPS data of Xi’an, and the result shows that prediction accuracy of proposed model is much more higher than the traditional models such as ARIMA, CNN and LSTM.