Regression Analysis

Regression analysis is a statistical tool that is used to identify relationships between independent and dependent variables. A cause and effect relationship is usually sought when using regression analysis. Regression analysis was coined by statistics genius Francis Galton, by accident when he was attempting to describe a biological phenomenon. He believed that the height of the offspring of tall parents would regress toward a more normal average. This, he called, regression toward the mean. In regression, the independent variable is hypothesized to affect the dependent variable. With regression analysis, more specifically, one is able to understand how the value of the dependent variable changes when the independent variable is manipulated or changed. The purpose of regression analysis is to predict or forecast. It is the most widely used method for studying quantitative evidence among the social sciences.

In economics, correlations are relatively common but identifying whether or not the correlation between the variables is a causal relationship is not so easy. This is why regression analysis is so widely used in the economic sector. Regressions can, in addition to ascertaining a causal relationship, identify how close and well determined the relationship is. Almost all empirical economic studies include regressions. The concept of regression analysis is relatively easy to understand and can be applied to data at hand. This method provides individuals with a sound basis for examining data that has been observed. It is outstandingly useful and can be used for both prediction of outcomes and descriptions of a large variety of data sets (Berk, 2003).

Regression analysis cannot be interpreted as a procedure for establishing a cause and effect relationship between variables. It can only indicate to what extent the two variables are associated. Any conclusions about cause and effect must be based on the judgment of the individual or individuals most knowledgeable about the application (Anderson, Sweeney & Williams, 2004).

Simple Regression

Simple regression refers to regression analysis with a single explanatory variable. Before regression analysis is possible, one has to make a hypothesis about the relationship that exists between the variables. For example, one may hypothesize that the more education an individual obtains, the higher their yearly earnings will be. Simple linear regression analysis involves one independent variable and one dependent variable in which the relationship between the variables is approximated by a straight line. The graph of the estimated simple linear regression equation is called the estimated regression line.

The graph is usually in the form of a scatter plot or scatter diagram. Scatter diagrams are constructed with the values of the independent variable x on the horizontal axis and the values of the dependent variable y on the vertical axis. When a scatter diagram is completed, it allows an individual to visibly and preliminarily draw conclusions about the possible relationship between the variables. In order to obtain the points included in a scatter point one must first develop a regression analysis equation which is done by using a procedure called the least squares method.

Coefficients

The correlation coefficient is a descriptive measure of the strength of linear association between the two variables, x and y. Values of the coefficient are always between