Regression in the Real World
I recently guest lectured for an applied regression analysis course at Penn State. Now, before you begin making certain assumptions—because as any statistician will tell you, assumptions are important in regression—you should know that I have no teaching experience whatsoever, and I’m not much older than the students I addressed.
I’m just 5 years removed from my undergraduate days at Virginia Tech, so standing behind the podium felt backwards. But it certainly provoked fond memories of my days in Hutcheson Hall—home to Virginia Tech’s Department of Statistics—learning the same concepts these students are learning.
I remember what it was like to be in their shoes. For instance, I had no idea how calculating an R-squared value by hand could meaningfully contribute to my life after college—especially when software like Minitab can so easily do the work for me.
Now I know better.
I wanted to show these students how regression and other statistical concepts could be applied in their future lines of work, and how integral a role statisticians can play in any business. As a student, I failed to grasp this because I was more concerned with the letter grade I needed to obtain to maintain a high GPA. Giving this guest lecture was an opportunity to renounce my old, flawed mentality.
What 5 years in the real world has taught me
Since graduating, I’ve engaged with many Minitab users, and I’ve worked with Minitab’s statistical consultants, who address a variety of business requests for statistical help. I’ve encountered numerous practical applications of how the tools in Minitab help people who need to analyze their data—many of whom lack formal statistical training—select the proper analysis, draw useful conclusions, and make key business decisions that lead to improved processes and increased profits.
If you want to draw actionable conclusions from your data, selecting and using the appropriate statistical tool is half the battle! In so many cases, people choose the wrong tools because they misunderstand what kind of data they have, or they haven’t fully defined the problem they are trying to solve. Questions like, ‘Are my data categorical or continuous,’ or ‘Are my observations independent,’ or ‘How are my data distributed,’ and ‘Does this analysis assume a particular distribution?’ are often difficult to answer, making selection of a statistical tool even more intimidating.
Fortunately, Minitab makes this entire process easy for professionals with any level of statistical expertise—our software provides the proper tool belt for solving statistical problems, including tools in the Assistant menu which offer detailed guidance to help the professional confidently choose the right analyses and make informed decisions for their business.
But formal training in statistical methods can give students a big advantage in the workplace. And regression—a very practical tool for modeling data and predicting outcomes—was my platform for communicating this idea, based on the experiences of some of Minitab’s own consulting statisticians.
What problems can regression solve?
If the data were accessible, I could use a regression analysis to show one thing that hasn’t changed about college—early morning classes have awful attendance rates, no matter what the subject is. My first lecture was during a Friday 8:00 a.m. section; the room was about half empty. And of those who actually attended, about half drowsily filtered in during the first 10 minutes of class.
But as I talked, I could practically see the synapses firing for some of those students. I presented several examples where regression came to the rescue of businesses in real world settings. These businesses had particular questions, and Minitab’s statistical consultants used regression to provide the answers.
I showed the students how:
- A pharmaceutical company used regression to assess the stability of the active ingredient in a drug to predict its shelf life in order to meet FDA regulations and identify a suitable expiration date for the drug.
- A credit card company applied regression analysis to predict monthly gift card sales and improve yearly revenue projections.
- A hotel franchise used regression to identify a profile for and predict potential clients who might default on a timeshare loan in order to reduce loan qualification rates among high-risk clients, adjust interest rates based on client risk factors, and minimize company losses.
- An insurance company used regression to determine the likelihood of a true problem existing when a home insurance claim was filed, in order to discourage customers from filing excessive or petty claims.
The results of these regression analyses, along with help from Minitab’s statistical consultants, gave these companies the confidence to make decisions they knew would improve their business. They were encouraged to identify solutions to address problem areas, and to implement new processes within their organization as well as new strategies to promote products and services to their clientele—knowing they could collect data and use the same tools again in the future to prove that changes were impactful.
The moral of the story
In a world filled with data, students who learn statistics leave college with a skill set that is highly sought after. When it comes to working with data, they will have advantages over professionals who don’t have formal statistical training.
But as most of us know, there’s more to data analysis than memorizing a bunch of equations. And even experienced statisticians can forget some of the nuances involved in analyses they haven’t used in a while. What they don’t forget is how to attack a problem.
In the end, that’s what I wanted these students to understand—that data-driven questions really boil down to problem-solving. What question am I trying to answer, and how do I tackle it?
The real world is about identifying a problem when we encounter it, choosing the right tool to solve it, and interpreting the answer in a way that drives a manager or executive to enact change within a business. Because businesses face real questions and challenges that statisticians—and software like Minitab—can help answer.