Which of the following is a potential drawback of using robust regression methods?
They always result in models with lower predictive accuracy than OLS regression
They can be computationally more expensive than OLS regression
They always require data normalization before model fitting
They are not applicable to datasets with categorical variables
What is a potential drawback of removing a highly correlated independent variable to deal with multicollinearity?
It may result in a loss of valuable information and reduce the model's accuracy.
It may improve the model's overall fit but reduce its interpretability.
It has no drawbacks and is always the best solution.
It may lead to an increase in the model's complexity.
You are comparing two linear regression models for predicting house prices. Model A has a lower RMSE than Model B. What does this imply about their predictive performance?
Model B is definitely overfitting the data.
Model A, on average, has smaller prediction errors than Model B.
Model A has a higher R-squared value than Model B.
Model A is guaranteed to make better predictions on all new data points.
If a linear regression model has an Adjusted R-squared value of 0.85, what does it indicate about the goodness of fit?
The model's predictions will be accurate 85% of the time.
The model is overfitting the data.
The model explains 15% of the variation in the dependent variable.
The model explains 85% of the variation in the dependent variable, accounting for the number of predictors.
Which metric is in the same units as the dependent variable, making it easier to interpret directly?
MAE
RMSE
R-squared
Adjusted R-squared
What does heteroscedasticity refer to in the context of multiple linear regression?
The presence of outliers in the data.
Non-constant variance of errors across different levels of the predictor variables.
Multicollinearity among the predictor variables.
Non-linearity in the relationship between predictors and outcome.
A model has a high R-squared but a low Adjusted R-squared. What is a likely explanation?
The model has high bias.
The model is overfitting.
The model is too simple.
The model is a perfect fit.
How does stepwise selection work in feature selection?
It uses L1 or L2 regularization to shrink irrelevant feature coefficients to zero.
It transforms the original features into a lower-dimensional space while preserving important information.
It ranks features based on their correlation with the target variable and selects the top-k features.
It iteratively adds or removes features based on a statistical criterion, aiming to find the best subset.
How does Lasso Regression differ from Ridge Regression in terms of feature selection?
Both Lasso and Ridge Regression can shrink coefficients to zero, but Lasso does it more aggressively.
Ridge Regression tends to shrink all coefficients towards zero but rarely sets them exactly to zero.
Lasso Regression can shrink coefficients to exactly zero, effectively performing feature selection.
Neither Lasso nor Ridge Regression performs feature selection; they only shrink coefficients.
Which technique is particularly useful for feature selection when dealing with high-dimensional datasets where the number of features exceeds the number of samples?
Ordinary Least Squares Regression
Lasso Regression
Elastic Net Regression
Ridge Regression