We study in this paper a forecasting model for long-term rates based both on the arbitrage-free hypothesis and the agents' rationality. The long-term rate is expressed as an average of expected short-term rates, which are modelized according to three models: two univariate models (with stationary and non-stationary rates) and one model which specifies the long-term anchor for the short-term rate as a function of the agents' expectations. Theses approaches are used to study French and German long-term rates between 1960 and 1996. We find that the model based on agents' expectations gives the best forecasts, especially for short-term horizons.