The Kalman filter is used to derive updating equations for the Bayesian data density in discrete time linear regression models with stochastic regressors. The implied “Bayes model” has time varying parameters and conditionally heterogeneous error variances. A σ-finite Bayes model measure is given and used to produce a new-model-selection criterion (PIC) and objective posterior odds tests for sharp null hypotheses like the presence of a unit root. This extends earlier work by Phillips and Ploberger [18]. Autoregressive-moving average (ARMA) models are considered, and a general test of trend-stationarity versus difference stationarity is developed in ARMA models that allow for automatic order selection of the stochastic regressors and the degree of the deterministic trend. The tests are completely consistent in that both type I and type II errors tend to zero as the sample size tends to infinity. Simulation results and an empirical application are reported. The simulations show that the PIC works very well and is generally superior to the Schwarz BIC criterion, even in stationary systems. Empirical application of our methods to the Nelson-Plosser [11] series show that three series (unemployment, industrial production, and the money stock) are level- or trend-stationary. The other eleven series are found to be stochastically nonstationary.