# Simple Linear Regression

Ordinary Least Squares (OLS) is a statistical method that produces a best-fit line between some outcome variable $$Y$$ and any number of predictor variables $$X_1, X_2, X_3, ...$$. These predictor variables may also be called independent variables or right-hand-side variables.

## Keep in Mind

• OLS assumes that you have specified a true linear relationship.
• OLS results are not guaranteed to have a causal interpretation. Just because OLS estimates a positive relationship between $$X_1$$ and $$Y$$ does not necessarily mean that an increase in $$X_1$$ will cause $$Y$$ to increase.
• OLS does not require that your variables follow a normal distribution.

# Implementations

## Gretl

# Load auto data
open https://github.com/LOST-STATS/lost-stats.github.io/blob/master/Data/auto.gdt

# Run OLS using the auto data, with mpg as the outcome variable
# and headroom, trunk, and weight as predictors
ols mpg const headroom trunk weight


## Matlab

% Load auto data

% Run OLS using the auto data, with mpg as the outcome variable
% and headroom, trunk, and weight as predictors

X = [intercept headroom trunk weight];
[b,bint,r,rint,stats] = regress(mpg,X);


## Python

# Use 'pip install statsmodels' or 'conda install statsmodels'
# on the command line to install the statsmodels package.

# Import the relevant parts of the package:
import statsmodels.api as sm
import statsmodels.formula.api as smf

# Get the mtcars example dataset
mtcars = sm.datasets.get_rdataset("mtcars").data

# Fit OLS regression model to mtcars
ols = smf.ols(formula='mpg ~ cyl + hp + wt', data=mtcars).fit()

# Look at the OLS results
print(ols.summary())



## R

# Load Data
# data(mtcars) ## Optional: automatically loaded anyway

# Run OLS using the mtcars data, with mpg as the outcome variable
# and cyl, hp, and wt as predictors
olsmodel <- lm(mpg ~ cyl + hp + wt, data = mtcars)

# Look at the results
summary(olsmodel)


## SAS

/* Load Data */
proc import datafile="C:mtcars.dbf"
out=fromr dbms=dbf;
run;
/* OLS regression */
proc reg;
model mpg = cyl hp wt;
run;


## Stata

* Load auto data
sysuse https://github.com/LOST-STATS/lost-stats.github.io/blob/master/Data/auto.dta

* Run OLS using the auto data, with mpg as the outcome variable
* and headroom, trunk, and weight as predictors