What is the primary difference between L1 and L2 regularization in the context of feature selection?
L1 regularization is less effective when dealing with highly correlated features compared to L2 regularization.
L2 regularization is more computationally expensive than L1 regularization.
L1 regularization can shrink some feature coefficients to exactly zero, performing feature selection, while L2 regularization generally shrinks coefficients towards zero without making them exactly zero.
L2 regularization forces the model to use all available features, while L1 regularization selects a subset of features.
What happens to the bias and variance of a linear regression model as the regularization parameter (lambda) increases?
Bias increases, Variance increases
Bias increases, Variance decreases
Bias decreases, Variance increases
Bias decreases, Variance decreases
Which metric is in the same units as the dependent variable, making it easier to interpret directly?
R-squared
Adjusted R-squared
RMSE
MAE
Logistic regression, a specific type of GLM, is best suited for modeling which type of response variable?
Time-to-event data
Count data
Continuous
Binary (two categories)
What is the primary goal of regularization techniques in linear regression?
To prevent overfitting by adding a penalty to the complexity of the model.
To speed up the training process of the linear regression model.
To handle missing data points in the dataset more effectively.
To improve model interpretability by selecting only the most relevant features.
What is the primary advantage of using Adjusted R-squared over R-squared when evaluating linear regression models?
Adjusted R-squared always increases when new predictors are added.
Adjusted R-squared penalizes the inclusion of irrelevant variables.
Adjusted R-squared is less sensitive to outliers compared to R-squared.
Adjusted R-squared is easier to interpret than R-squared.
When using Principal Component Analysis (PCA) as a remedy for multicollinearity, what is the primary aim?
To introduce non-linearity into the model
To remove all independent variables from the model
To increase the sample size of the dataset
To create new, uncorrelated variables from the original correlated ones
Which of the following is a potential drawback of using robust regression methods?
They can be computationally more expensive than OLS regression
They are not applicable to datasets with categorical variables
They always require data normalization before model fitting
They always result in models with lower predictive accuracy than OLS regression
Elastic Net Regression combines the penalties of which two regularization techniques?
Lasso Regression and Ridge Regression
Ridge Regression and Polynomial Regression
Linear Regression and Ridge Regression
Lasso Regression and Polynomial Regression
Why is evaluating the model on a separate test set crucial in Polynomial Regression?
To visualize the residuals and check for any non-linear patterns.
To calculate the model's complexity and determine the optimal degree of the polynomial.
To fine-tune the model's hyperparameters and improve its fit on the training data.
To estimate the model's performance on unseen data and assess its generalization ability.