R Programming for Linear Regression: A Data Analysis Guide

We’re all trying to find patterns in the world. Does spending more on advertising lead to more sales? Does studying longer *really* improve exam scores? Does a person’s height have any relationship to their weight? We intuitively feel these things are connected, but how can we prove it? And more importantly, how can we *quantify* that connection? This is where the power of statistical modeling comes in, and one of the most fundamental and useful tools in this field is Linear Regression. If you’re working with data in R, understanding linear regression is like learning the first, most important chord on a guitar-it’s the foundation for so much more.

In simple terms, linear regression is a way to model the relationship between two variables by fitting a straight line to the observed data. One variable is considered the predictor variable (or independent variable), which we’ll call ‘x’. The other is the response variable (or dependent variable), which we’ll call ‘y’. Our goal is to find the *one* specific line that best describes the relationship between them.

The simple math behind the magic line 🔗

If you remember back to high school math, the equation for any straight line is $y = a + b \times x$. It turns out, this simple equation is the entire engine of linear regression. The magic isn’t in the equation itself, but in how we find the *best* values for ‘a’ and ‘b’ based on our data. Let’s break down those two key pieces, called coefficients.

The intercept (a): Where the line starts 🔗

The intercept, often labeled as ‘a’ (or $\beta_0$ in many textbooks), is the value of ‘y’ when ‘x’ is zero. It’s the point where the regression line crosses the vertical y-axis. Think of it as the baseline value. If you’re modeling the relationship between advertising spend (‘x’) and sales (‘y’), the intercept would be your *predicted sales* if you spent zero dollars on advertising. In many real-world scenarios, the intercept isn’t the most interesting part, but it’s a crucial piece that anchors the line.

The slope (b): The engine of the relationship 🔗

The slope, ‘b’ (or $\beta_1$), is the most exciting part. It quantifies the change in ‘y’ for every one-unit change in ‘x’. If the slope is 2, it means that for every 1-unit increase in ‘x’, ‘y’ is predicted to increase by 2 units. If the slope is -0.5, it means for every 1-unit increase in ‘x’, ‘y’ is predicted to *decrease* by 0.5 units. This single number tells us both the direction (positive or negative) and the strength of the relationship. When you hear that R “calculates” the model, it’s using a method called Ordinary Least Squares (OLS) to find the exact ‘a’ and ‘b’ values that create a line that is as close as possible to all the data points simultaneously.

Building your first regression model in R 🔗

Now, let’s get our hands dirty. R makes building a linear model incredibly straightforward with its core `lm()` function, which stands for “linear model.” The basic syntax is model <- lm(formula, data).

The formula: Speaking R’s language 🔗

The formula is the most important part. R uses a special tilde symbol `~` to mean “is predicted by.” So, the formula y ~ x is R’s way of saying “we want to predict ‘y’ using ‘x’.” If we were using our height and weight example, and our columns were named `weight_kg` and `height_cm`, our formula would be weight_kg ~ height_cm.

A practical example: Height and weight 🔗

Let’s imagine we have a dataset in R called `people_data`. This data frame has two columns: `height_cm` and `weight_kg`.

To build a model that predicts a person’s weight based on their height, we would run this single line of code:

model_fit <- lm(weight_kg ~ height_cm, data = people_data)

That’s it. In one command, R has done all the complex math. It has analyzed all the data points and found the best-fitting intercept (‘a’) and slope (‘b’) for our line. The results are now stored in the object we named `model_fit`.

Did our model actually work? Interpreting the summary 🔗

Creating the model is easy. The real skill of a data analyst is understanding if the model is any good. To do this, we use the `summary()` function:

summary(model_fit)

This command prints a lot of information, which can be intimidating. Let’s focus on the most important parts.

Residuals: The model’s “errors” 🔗

The summary first shows a five-point summary (Min, 1Q, Median, 3Q, Max) of the residuals. A residual is simply the difference between an *actual* observed value (a real person’s weight in our data) and the *predicted* value from our model (the weight our line predicted for that height). It’s the “error” for that data point.

In a good model, the residuals should be scattered randomly around zero. What we look for here is a Median value very close to 0 and 1Q/3Q values that are roughly symmetrical. It’s a quick check to see if the model seems biased.

Coefficients: The values for ‘a’ and ‘b’ 🔗

This is the main event. You’ll see a table with rows for `(Intercept)` and your ‘x’ variable (`height_cm`). The `Estimate` column gives you the calculated values for ‘a’ and ‘b’.

• (Intercept) Estimate: This is your ‘a’ value.
• height_cm Estimate: This is your ‘b’ value, the slope. If this number is 0.8, it means our model predicts that a 1 cm increase in height is associated with a 0.8 kg increase in weight.

The most important columns here are the p-values (labeled `Pr(>|t|)`). A p-value is a measure of statistical significance. The common rule of thumb is that if the p-value is very small (e.g., less than 0.05, often marked with asterisks like `***`), it means the variable is statistically significant. It suggests that the ‘x’ variable (height) has a real, non-zero effect on the ‘y’ variable (weight) and isn’t just there by random chance.

R-squared: How much does our model explain? 🔗

Near the bottom, you’ll find the Multiple R-squared and Adjusted R-squared values. This number, from 0 to 1, tells you how much of the variance in your ‘y’ variable is *explained* by your ‘x’ variable. An R-squared of 0.75, for example, would mean that 75% of the variation in people’s weights (in our dataset) can be explained by their height. The other 25% is due to other factors (genetics, diet, exercise, etc.). A higher R-squared is often better, but what’s considered “good” depends entirely on the field of study.

Making predictions with the predict() function 🔗

So, we have a model that seems significant. What do we do with it? We predict things! This is where the `predict()` function comes in. Its job is to take our trained model and apply it to new data.

The key is that we must give it a new data frame with the *exact same column name* as our predictor. If we want to predict the weight for someone who is 175 cm tall, we would do this:

First, create the new data:

new_heights <- data.frame(height_cm = 175)

Then, run the prediction:

predicted_weight <- predict(model_fit, newdata = new_heights)

The `predicted_weight` object will now hold the model’s best guess for the weight of a 175 cm person, based on the pattern it learned from the original data.

Visualizing linear regression in R 🔗

Finally, one of the best ways to know if your model makes sense is to look at it. We can easily plot our data and our regression line.

Creating the scatter plot 🔗

First, we plot our raw data. The `plot()` function is perfect for this. This command creates a scatter plot of all our data points:

plot(people_data$height_cm, people_data$weight_kg, main=”Height vs. Weight”, xlab=”Height (cm)”, ylab=”Weight (kg)”)

Adding the regression line 🔗

Now, while that plot is open, we can magically overlay our regression line using the `abline()` function, which stands for “add line.”

abline(model_fit, col = “red”, lwd = 3)

This command tells R to take the intercept (‘a’) and slope (‘b’) from our `model_fit` object and draw the corresponding line on the plot. We’ve made it red (`col = “red”`) and a bit thicker (`lwd = 3`) to stand out.

[Image: A scatter plot of height and weight data points with a red linear regression line (abline) drawn through them.]

Seeing that red line slice through the cloud of data points is often the “a-ha!” moment. It’s the visual representation of the entire relationship we’ve just modeled, moving from abstract statistical concepts to a simple, intuitive line.

What do you think? How could you use a simple linear regression model to explore a relationship in your own field of work or a hobby you’re passionate about? We discussed R-squared, but what do you think are the dangers of relying on just one number to decide if a model is “good”?

References

1. https://www.scribbr.com/statistics/ordinary-least-squares/
2. https://www.geeksforgeeks.org/interpreting-the-results-of-linear-regression-using-r/
3. https://www.scribbr.com/statistics/r-squared/
4. https://www.investopedia.com/terms/l/linearregression.asp