Let G be a simple connected graph. The terminal distance matrix of G is the distance matrix between all pendant vertices of G. In this paper, we study the terminal distance matrix and compute the characteristic polynomial of this matrix for some rooted trees. Also we obtain lower bounds for the spectral radius of the terminal distance matrix of graphs and characterize those graphs for which the bounds are best possible.