To increase the computing speed of neural networks by means of parallel performance, a new mode of neural network, named Tri-state Cascading Pulse Coupled Neural Network (TCPCNN), is presented in this paper, which takes the ideas of three-state and pipelining used in circuit designing into neural network, and creates new neuron with three states: sub-firing, firing and inhibition. The proposed model can transmit signals in parallel way, as it is inspired not only in the direction of auto-wave propagation but also in its transverse direction in neural network. In this paper, TCPCNN is applied to find the shortest path, and the experimental results indicate that the algorithm has lower computational complexity, higher accuracy, and secured full-scale searching. Furthermore, it has little dependence on initial conditions and parameters. The algorithm is tested by some experiments, and its results are compared with some other classical algorithms --- Dijkstra algorithm, Bellman-Ford algorithm and a new algorithm using pulse coupled neural networks.