What happens to the bias and variance of a linear regression model as the regularization parameter (lambda) increases?
Bias increases, Variance decreases
Bias decreases, Variance decreases
Bias increases, Variance increases
Bias decreases, Variance increases
What hyperparameter controls the strength of regularization in Ridge, Lasso, and Elastic Net Regression?
Learning Rate
Regularization Parameter
Number of Iterations
Tolerance
Which of the following is NOT a common link function used in Generalized Linear Models?
Logit
Probit
Polynomial
Complementary log-log
What does a high Cook's distance value indicate?
The observation has high leverage but low influence
The observation has both high leverage and high influence
The observation has low leverage but high influence
The observation is not an outlier
Which of the following is a potential drawback of using robust regression methods?
They are not applicable to datasets with categorical variables
They always require data normalization before model fitting
They can be computationally more expensive than OLS regression
They always result in models with lower predictive accuracy than OLS regression
If a linear regression model has an Adjusted R-squared value of 0.85, what does it indicate about the goodness of fit?
The model is overfitting the data.
The model explains 85% of the variation in the dependent variable, accounting for the number of predictors.
The model explains 15% of the variation in the dependent variable.
The model's predictions will be accurate 85% of the time.
If a predictor has a p-value of 0.02 in a multiple linear regression model, what can you conclude?
The predictor explains 2% of the variance in the outcome.
The predictor has a practically significant effect on the outcome.
The predictor is not statistically significant.
The predictor is statistically significant at the 0.05 level.
Which technique is particularly useful for feature selection when dealing with high-dimensional datasets where the number of features exceeds the number of samples?
Elastic Net Regression
Ordinary Least Squares Regression
Lasso Regression
Ridge Regression
How do Generalized Linear Models (GLMs) extend the capabilities of linear regression?
By allowing only categorical predictor variables.
By assuming a strictly linear relationship between the response and predictor variables.
By enabling the response variable to follow different distributions beyond just normal distribution.
By limiting the analysis to datasets with a small number of observations.
What is the primary difference between L1 and L2 regularization in the context of feature selection?
L1 regularization can shrink some feature coefficients to exactly zero, performing feature selection, while L2 regularization generally shrinks coefficients towards zero without making them exactly zero.
L2 regularization forces the model to use all available features, while L1 regularization selects a subset of features.
L1 regularization is less effective when dealing with highly correlated features compared to L2 regularization.
L2 regularization is more computationally expensive than L1 regularization.