Linear Regression

Salient Notes

1. Linear Regression is a method of establishing relationship between a set of variables (called independent variables or predictors) and a dependent or the target variable, called the outcome. e.g., in the case of a house, price is the outcome/dependent variable, and area, # of bedrooms, locality etc. are the independent or predictor variables.
2. Explains the change in the outcome variable based on changes in the predictor variables.
• Simple Linear Regression: Only one predictor
• Multiple Linear Regression: More than one predictor.
3. Uses: Forecasting & Prediction. 
• Substantial overlap with each other
• Linear Regression guarantees interpolation, not extrapolation.
• Important to know when to do Forecast and when Prediction.
4. LR only shows correlation, not causation. In restrictive settings such as medicine, it could show causation.
5. LR is not the only technique of regression. LR is one form a parametric regression in that you work with a fixed set of predictor variables. There are also non-parametric regression techniques where there is no fixed set of predictor variables or parameters. 

Resources

- https://stats.stackexchange.com/questions/268638/confusion-about-parametric-and-non-parametric-model

- https://machinelearningmastery.com/parametric-and-nonparametric-machine-learning-algorithms/