Scribd
Upload
17 views

Bayesian Multiple Linear Regression

The document discusses Bayesian multiple linear regression as a method for predicting food spending based on various factors, highlighting its advantages in handling uncertainty and incorporating prior knowledge. However, it argues that given the large dataset size and observed heteroskedasticity, Ordinary Least Squares (OLS) regression with robust standard errors is a more suitable approach. The document emphasizes the computational demands of Bayesian methods and suggests alternatives like Generalized Least Squares (GLS) for addressing heteroskedasticity.

Uploaded by

josefvillacampa
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
17 views

Bayesian Multiple Linear Regression

The document discusses Bayesian multiple linear regression as a method for predicting food spending based on various factors, highlighting its advantages in handling uncertainty and incorporating prior knowledge. However, it argues that given the large dataset size and observed heteroskedasticity, Ordinary Least Squares (OLS) regression with robust standard errors is a more suitable approach. The document emphasizes the computational demands of Bayesian methods and suggests alternatives like Generalized Least Squares (GLS) for addressing heteroskedasticity.

Uploaded by

josefvillacampa
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 7

Bayesian Multiple Linear Regression

Imagine you're trying to predict how much money a family spends on food (LOGFOODEXP) based on several
factors like their income (LOGHINC), household size (HSIZE), and age of the household head (HHAGE).

Traditional Regression gives you a single estimate for how each factor affects food spending. It's like getting a
single number that says, "For every extra dollar in income, food spending increases by this much." Bayesian
Regression does the same thing but also tells you how confident you should be in those estimates. It's like getting a
range of possible numbers instead of just one, so you can see how likely each possible effect is.

Why Use Bayesian Regression?

Uncertainty: It helps you understand how sure you are about your predictions. This is important because real-world
data can be messy and uncertain.

Prior Knowledge: If you already know something about the problem like how income affects spending. Bayesian
regression lets you use that knowledge to improve your predictions.

Small Data: It works well even when you don't have a lot of data, which is common in many fields.

Suppose you want to predict food spending based on income. Bayesian regression might tell you: "For every extra
dollar in income, food spending probably increases by between $0.20 and $0.50." This range shows the uncertainty
in the estimate. It's like having a more nuanced understanding of how things work, which can be really helpful in
making decisions.

Assumptions

1. Linearity: Linear relationship between dependent and independent variables.


2. Normality of Errors: Errors follow a normal distribution.
3. Independence of Errors: Errors are independent across observations.
4. No Multicollinearity: Predictors are not highly correlated.
5. Homoscedasticity: Constant variance of errors.
6. Valid Priors: Reasonable prior distributions for parameters.
7. Fixed Predictors: Predictors are fixed constants.
8. Sufficient Data: Enough data to update priors meaningfully.
9. Correct Model Specification: Likelihood function correctly represents the data-generating process.

DIAGNOSTIC TESTS

1. Linearity of variables: Scatterplots and Partial residual regression plots

2. Normality of Errors: histogram of residuals, qnorm residuals.

3. Independence of Errors: Durbin-Watson test to check for autocorrelation in residuals. Variance of


residuals
is constant across predicted values.
4. No Multicollinearity: Variance inflation factors estat vif

5. Homoscedasticity: Constant variance of errors. Breusch-Pagan test: (estat hottest)

6. Valid Priors:
7. Fixed Predictors: Predictors are fixed constants.

8. Sufficient Data: Enough data to update priors meaningfully.

9. Correct Model Specification: Likelihood function correctly represents the data-generating process.

Given the size of the dataset (147,717 observations) and the issues with heteroskedasticity
observed in the OLS regression, Bayesian multiple regression may not be the most suitable
approach for this analysis. Bayesian methods, while offering flexibility and the ability to
incorporate prior knowledge, can be computationally intensive, especially when dealing with
large datasets. Markov Chain Monte Carlo (MCMC) sampling, which is commonly used in
Bayesian regression, can become inefficient as the dataset size increases, resulting in longer
fitting times and potential convergence issues (Gelman et al., 2013). This could present
challenges when working with large datasets like the one in question, where computational
resources may become a limiting factor.

In addition, heteroskedasticity, which violates the homoscedasticity assumption in ordinary least


squares (OLS) regression, could also pose challenges to the assumptions of Bayesian regression.
In Bayesian models, error variance is assumed to be constant across observations. If
heteroskedasticity is present, it could bias the model results or render them less reliable
(Kruschke, 2015). While Bayesian regression offers flexibility in modeling uncertainty, the
assumption of homoscedasticity remains important. In situations where heteroskedasticity is
observed, alternative methods such as Generalized Least Squares (GLS) or heteroskedasticity-
robust standard errors may provide more reliable results. These methods are specifically
designed to address non-constant error variance, making them more appropriate for datasets
exhibiting heteroskedasticity (White, 1980).

In this instance, I believe that OLS (Ordinary Least Squares) is a better fit for my research,
given the large dataset (147,717 observations) and the issue of heteroskedasticity observed in the
initial OLS regression. Despite the heteroskedasticity, OLS remains a widely used method for
regression analysis due to its simplicity, efficiency, and the availability of robust methods that
can address violations of assumptions, such as heteroskedasticity (Wooldridge, 2010). For large
datasets, OLS can still provide reliable and efficient estimates when robust standard errors are
applied, which allows for valid inference even when heteroskedasticity is present (White, 1980).

The heteroskedasticity in my data, which violates the assumption of constant error variance in
OLS, can be addressed using White's heteroskedasticity-consistent standard errors estimator,
ensuring valid parameter estimates without transforming the model (White, 1980). Alternatively,
Generalized Least Squares (GLS) could be employed if the heteroskedasticity is suspected to
be systematic, though OLS with robust standard errors remains a simpler and effective solution
for addressing this issue.
While Bayesian methods offer flexibility in modeling uncertainty and prior information, they are
computationally demanding, particularly with large datasets. According to Gelman et al. (2013),
Bayesian models require substantial computational resources due to the use of Markov Chain
Monte Carlo (MCMC) sampling, which may not be efficient for datasets of this size. Moreover,
the need to specify priors and interpret the posterior distributions in Bayesian analysis can add
unnecessary complexity, and may not yield substantial advantages over OLS when working with
large, well-behaved datasets.

reference:

Therefore, unless there is a strong requirement for prior information or complex uncertainty
modeling, I find that OLS with robust standard errors or GLS is a more efficient and practical
choice for my research.

Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013).
Bayesian data analysis (3rd ed.). CRC Press.

White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test


for heteroskedasticity. Econometrica, 48(4), 817–838. https://doi.org/10.2307/1912934

Wooldridge, J. M. (2010). Econometric analysis of cross section and panel data (2nd ed.). MIT
Press.

Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013).
Bayesian data analysis (3rd ed.). CRC Press.

Kruschke, J. K. (2015). Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan (2nd
ed.). Academic Press.

White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test


for heteroskedasticity. Econometrica, 48(4), 817–838. https://doi.org/10.2307/1912934
BAYESIAN RESULTS FOR FIES 2018

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy