The changes of runoff in the middle reaches of the Yellow River basin of China have received considerable attention owing to their sharply decline during recent decades. In this paper, the impacts of rainfall characteristics and land use and cover change on water yields in the Jingle sub-basin of the middle reaches of the Yellow River basin were investigated using a combination of statistical analysis and hydrological simulations. The Levenberg Marquardt and Analysis of Variance methods were used to construct multivariate, nonlinear, model equations between runoff coefficient and rainfall intensity and vegetation coverage. The land use changes from 1971 to 2017 were ascertained using transition matrix analysis. The impact of land use on water yields was estimated using the M-EIES hydrological model. The results show that the runoff during flood season (July to September) decreased significantly after 2000, whereas slightly decreasing trend was detected for precipitation. Furthermore, there were increase in short, intense, rainfall events after 2000 and this rainfall events were more conducive to flood generation. The “Grain for Green” project was carried out in 1999, and the land use in the middle reaches of the Yellow River improved significantly, which make the vegetation coverage (Vc) of the Jingle sub-basin increased by 13%. When Vc approaches 48%, the runoff coefficient decreased to the lowest, and the vegetation conditions have the greatest effect on reducing runoff. Both land use and climate can change the water yield in the basin, but for areas where land use has significantly improved, the impact of land use change on water yield plays a dominant role. The results acquired in this study provide a useful reference for water resources planning and soil and water conservation in the erodible areas of the middle reaches of the Yellow River basin.