dcsimg
 

What Is the Difference between Linear and Nonlinear Equations in Regression Analysis?

Fourier nonlinear functionPreviously, I’ve written about when to choose nonlinear regression and how to model curvature with both linear and nonlinear regression. Since then, I’ve received several comments expressing confusion about what differentiates nonlinear equations from linear equations. This confusion is understandable because both types can model curves.

So, if it’s not the ability to model a curve, what is the difference between a linear and nonlinear regression equation?

Linear Regression Equations

Linear regression requires a linear model. No surprise, right? But what does that really mean?

A model is linear when each term is either a constant or the product of a parameter and a predictor variable. A linear equation is constructed by adding the results for each term. This constrains the equation to just one basic form:

Response = constant + parameter * predictor + ... + parameter * predictor

Y = b o + b1X1 + b2X2 + ... + bkXk

In statistics, a regression equation (or function) is linear when it is linear in the parameters. While the equation must be linear in the parameters, you can transform the predictor variables in ways that produce curvature. For instance, you can include a squared variable to produce a U-shaped curve.

Y = b o + b1X1 + b2X12

This model is still linear in the parameters even though the predictor variable is squared. You can also use log and inverse functional forms that are linear in the parameters to produce different types of curves.

Here is an example of a linear regression model that uses a squared term to fit the curved relationship between BMI and body fat percentage.

Linear model with squared term

Nonlinear Regression Equations

While a linear equation has one basic form, nonlinear equations can take many different forms. The easiest way to determine whether an equation is nonlinear is to focus on the term “nonlinear” itself. Literally, it’s not linear. If the equation doesn’t meet the criteria above for a linear equation, it’s nonlinear.

That covers many different forms, which is why nonlinear regression provides the most flexible curve-fitting functionality. Here are several examples from Minitab’s nonlinear function catalog. Thetas represent the parameters and X represents the predictor in the nonlinear functions. Unlike linear regression, these functions can have more than one parameter per predictor variable.

Nonlinear function One possible shape
Power (convex): Theta1 * X^Theta2 Power function in nonlinear regression
Weibull growth: Theta1 + (Theta2 - Theta1) * exp(-Theta3 * X^Theta4) Weibull growth function in nonlinear regression
Fourier: Theta1 * cos(X + Theta4) + (Theta2 * cos(2*X + Theta4) + Theta3 Fourier function for nonlinear regression

Here is an example of a nonlinear regression model of the relationship between density and electron mobility.

Nonlinear regression model for electron mobility

The nonlinear equation is so long it that it doesn't fit on the graph:

Mobility = (1288.14 + 1491.08 * Density Ln + 583.238 * Density Ln^2 + 75.4167 * Density Ln^3) / (1 + 0.966295 * Density Ln + 0.397973 * Density Ln^2 + 0.0497273 * Density Ln^3)

Linear and nonlinear regression are actually named after the functional form of the models that each analysis accepts. I hope the distinction between linear and nonlinear equations is clearer and that you understand how it’s possible for linear regression to model curves! It also explains why you’ll see R-squared displayed for some curvilinear models even though it’s impossible to calculate R-squared for nonlinear regression.

If you're learning about regression, read my regression tutorial!

Comments

Name: Alan Slavin • Wednesday, August 13, 2014

Your blogs are very useful.
I am using a numerical calculation for the indiffusion of a gas into a metal, using discrete time steps where the gas concentration (with sample depth) at each step depends on the step before it. I want to test this model against actual data, with the goal of determining values for the fitting parameters in the model. Each term for a given time step is a linear function of the fitting parameters, but the single analytical diffusion equation that describes a related process is non-linear. Would my model be considered linear or non-linear? Thanks, Al


Name: Jim Frost • Tuesday, August 19, 2014

Hi Alan,

I'm not sure that I understand the data and problem well enough to answer your question. However, it sounds like the process may be locally linear when you look at small steps in time but nonlinear when you look at the process as a whole. This simply means that a linear relationship is correct for small differences but you need a nonlinear equation for larger differences.

In general, you shouldn't predict outside the range of your data. However, you can change the range based on how you perform the analysis. As the range changes, the relationship within that range can also change. In this case, it may be changing from linear to nonlinear.

I illustrate an example of this in my post about how to interpret the constant. Follow the link and scroll down to the section titled "Zero Settings for All of the Predictor Variables Can Be Outside the Data Range".

http://blog.minitab.com/blog/adventures-in-statistics/regression-analysis-how-to-interpret-the-constant-y-intercept

This is a very different scenario than what you describe but shows what I mean when I say that the relationship is locally linear for a smaller range (body weight and heights in this example) but that relationship changes if you expand the range.

As for whether it's considered linear or nonlinear? Again, I'd say yours is locally linear but the larger process is nonlinear. In terms of choosing which equation to use, you'll have to use your process knowledge as well as assess the fit of the different equations. I wrote a blog post that might help you there:

http://blog.minitab.com/blog/adventures-in-statistics/curve-fitting-with-linear-and-nonlinear-regression

If I've misunderstood the question that you were asking, please don't hesitate to write again! Thanks for reading!
Jim


blog comments powered by Disqus