Bayesian regression with heteroscedastic error density and parametric mean functionby Justinas Pelenis

Journal of Econometrics


Applied Mathematics / Economics and Econometrics / History and Philosophy of Science





Bayesian semi-parametrics

Bayesian conditional density estimation

Heteroscedastic linear regression mean function defined by a conditional moment condition and flexible predictor dependent error densities. Sufficient conditions to achieve posterior consistency of the regression parameters and conditional error densities are provided. In experiments, the proposedmethod compares favorably with classical and alternative Bayesian estimation methods for the estimation of the regression coefficients and conditional densities. © 2013 Elsevier B.V. All rights reserved. 1. Introduction

Estimation of regression coefficients in linear regression models can be consistent but inefficient if heteroscedasticity is ignored.

Furthermore, the regression curve only provides a summary of the mean effects but does not provide any information regarding conditional error distributions which might be of interest to the decision maker. Estimation of conditional error distributions is useful in settings where forecasting and out of sample predictions are the object of interest. In this paper we propose a novel

Bayesian method for consistent estimation of both linear regression coefficients and conditional residual distributions when data generating process satisfies a linear conditional moment restriction E[y|x] = x′β or amore general restricted conditional moment condition of E[y|x] = h(x, θ) for some known function h. The contribution of this proposal is that themodel is correctly specified for a large class of true data generating processes without imposing specific restrictions on the conditional error distributions. Hence consistent and efficient estimation of the parameters of interest might be expected.

The most widely used method to estimate the mean of a continuous response variable as a function of predictors is, without doubt, the linear regression model. Often the models considered impose the assumptions of constant variance and/or symmetric ✩ Previously circulated under the title of ‘‘Bayesian Semiparametric Regression’’.

E-mail address: and unimodal error distributions. Such restrictions are often inappropriate for real-life datasets where conditional variability, skewness and asymmetry might hold. The prediction intervals obtained using models with constant variance and/or symmetric error distributions are likely to be inferior to the prediction intervals obtained from models with predictor dependent residual densities.

To achieve full inference of parameters of interest and conditional error densitieswe propose a semi-parametric Bayesianmodelwith a parametric mean function and a non-parametric prior for covariate dependent error densities to simultaneously estimate both regression coefficients and the conditional error densities. A Bayesian approachmight bemore effective in small samples as it enables exact inference given observed data instead of relying on asymptotic approximations.

There is a substantial semi-parametric Bayesian literature that focuses on non-parametric priors for the mean function and parametric priors on the error distributions. An alternative common strand of the literature focuses on either parametric or nonparametric priors on the mean function and uses explicitly nonparametric priors on the error distributions and it is this choice of the priors on the error distributions that motivates the discussion in this manuscript. The common assumption is that the errors are generated independently from regressors x and usually satisfy either a median or quantile restriction. Estimation and consistency of such models is discussed in Kottas and Gelfand (2001), Hirano (2002), Amewou-Atisso et al. (2003), Conley et al. (2008) and Wu and Ghosal (2008) among others. However, estimation of the parameters and error densities under the assumption of independencemight be inconsistent if errors and covariates are dependent.Journal of Econometric

Contents lists availa

Journal of Ec journal homepage: www.el

Bayesian regression with heteroscedastic parametric mean function✩

Justinas Pelenis

Institute for Advanced Studies, Vienna, Austria a r t i c l e i n f o

Article history:

Received 6 April 2012

Received in revised form 8 May 2013

Accepted 2 October 2013

Available online 22 October 2013

JEL classification: a b s t r a c t

In this paper we consider Ba regression as a particular exa other semi-parametric mode the errors are independent f distributions should be prefe pendent of predictorsmight covariates are dependent. To0304-4076/$ – see front matter© 2013 Elsevier B.V. All rights reserved. 178 (2014) 624–638 le at ScienceDirect onometrics error density and yesian estimation of restricted conditional moment models with the linear mple. A common practice in the Bayesian literature for linear regression and ls is to use flexible families of distributions for the errors and to assume that rom covariates. However, a model with flexible covariate dependent error rred for the following reason. Assuming that the error distribution is indeead to inconsistent estimation of the parameters of interest when errors and address this issue,wedevelop a Bayesian regressionmodelwith a parametric

J. Pelenis / Journal of Econom

For example, under heteroscedasticity or conditional asymmetry of error distributions the pseudo-true values of regression coefficients in a linear model with errors generated by covariate independent mixtures of normals are not generally equal to the true parameter values.

One of the contributions of this paper is to show that the model proposed in thismanuscript that incorporates predictor dependent residual densities is flexible and leads to a consistent estimation of both parameters of interest θ and conditional error densities.

Other Bayesian proposals that incorporate predictor dependent residual density modeling into parametric models are by Pati and