What is linear regression?
Q: What is linear regression?
A: Linear regression is a way of looking at how something changes when other things change using math. It uses a dependent variable and one or more explanatory variables to create a straight line, known as the "line of regression".
Q: What are the advantages of linear regression?
A: The models which depend linearly on their unknown parameters are easier to fit than models which are non-linearly related to their parameters. Additionally, the statistical properties of the resulting estimators are easier to determine.
Q: What are some practical uses for linear regression?
A: Linear regression can be used to fit a predictive model to observed values (data) in order to make predictions, forecasts or reductions. It can also be used to quantify the strength of relationships between variables and identify subsets of data that contain redundant information about another variable.
Q: How do linear regression models try to minimize errors?
A: Linear regression models try to make the vertical distance between the line and the data points (the residuals) as small as possible. This is done by minimizing either the sum of squares of residuals (least squares), lack of fit in some other norm (least absolute deviations), or minimizing a penalized version of least squares loss function (ridge regression).
Q: Is it possible for linear regressions models not be based on least squares?
A: Yes, it is possible for linear regressions models not be based on least squares but instead use methods such as minimizing lack of fit in some other norm (least absolute deviations) or minimizing a penalized version of least squares loss function (ridge regression).
Q: Are “linear model” and “least squares” synonyms?
A: No, they are not synonyms. While they are closely linked, "linear model" refers specifically to using a straight line while "least squares" refers specifically to trying minimize errors by making sure that there is minimal vertical distance between the line and data points.
