What is the difference between HLM and OLS?

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

I am conducting a research whereby I have a few independent variables (all of them are dummies), moderators (one is a dummy, the other is continuous) and a continuous dependent variable.

I was told to use the ordinary least squares regression (OLS), but what is the difference between the OLS regression and a hierarchical linear regression analysis?

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏0 回帖

关键词：difference Between DIFFER erence What difference ordinary between least

There is some confusion regarding the term "hierarchical regression". Most often this indeed refers to multilevel models, as the previous poster indicated. In psychology textbooks (e.g., Cohen, Cohen, West, and Aiken), hierarchical regression refers to a simple OLS regression in which predictors are entered in some order (presumably based on theory) and then increments in explained variance and changes in regression coefficients are evaluated. In that sense "hierarchical regression" is not much different from OLS regression, other than certain sets of predictors are entered in the regression in a certain order. The similarity of the terms is somewhat unfortunate, because it creates some confusion.

Building hierarchical models is all about comparing groups. The power of the model is that you can treat the information about a particular group as evidence relating how that group compares to the aggregate behavior for a particular level, so if you don't have a lot of information about a single group, that group gets pushed towards the mean for the level. Here's an example:

Let's say we wanted to build a linear model describing student literacy (perhaps as a function of grade-level and socioeconomic status) for a region. What's the best way to go about this? One naive way would be to just treat all the students in the region as one big group and calculate an OLS model for literacy rates at each grade level. There's nothing exactly wrong with this, but let's say that for a particular student, we know that they attend an especially good school out in the burbs. Is it really fair to apply the county-wide average literacy for their grade to this student? Of course not, their literacy will probably be higher than average because of our observation about their school. So as an alternative, we could develop a separate model for each school. This is great for big schools, but again: what about those small private schools? If we only have 15 kids in a class, we're probably not going to have a very accurate model.

Hierarchical models allow us to do both simultaneously. At one level, we calculate the literacy rate for the entire region. At another level, we calculate the school-specific literacy rates. The less information we have about a particular school, the more closely it will approximate the across-school mean. This also allows us to step up the model to consider other school districts, and maybe even go a level higher to compare literacy between states or even consider differences between countries. Anything going on all the way up at the country level won't have a huge impact all the way down at the county level because there are so many levels in between, but information is information and we should allow it the opportunity to influence our results, especially where we have very little data.

So if we have very little data on a particular school, but we know how schools in that country, state, and county generally behave, we can make some informed inferences about that school and treat new information as evidence against our beliefs informed by the larger groups (the higher levels in the hierarchy).

Suppose you have a data sampled over N different geographical locations. At each location you take n measurements: each measurement, say, records the wind speed, which is dependent on the temperature and humidity (I have no idea about the real world, this is just for illustration purposes). The simple way (or ordinary linear regression) to proceed is to assume that the data is independent of the geographical location. In this case you just pool everything into one sample and perform OLR analysis. But if you have a hunch, that the location might have influence, you add another level of hierarchy - you assume that wind speed and humidity are dependent on geographical location. Hence, the first level of hierarchy is used to measure the variability within-source, or in each different geographical locations, and the second level takes into account the between-source variability.

You can add as many hierarchies as you need.

P.S. hierarchical regression, as performed within frequentist framework, is usually called multilevel regression, while within Bayesian formalism it is hierarchical regression.

I don't think this is completely true, because if you implement dummy variables in the OLS framework, you do not need to assume independence of the locations. In fact you can implement codings for as many different levels and features as you like, though that might be mathematically equivalent to whatever hierachical regression is in the OLS context (I dunno lol)

Keeping my answer short - one difference is when you are trying to predict results for effects that are related to your other independent variables, but you didn't observe in the sample. The simplest example of this is if there are particular domains where there is no direct information from the data you have - the classic example is that you have not sampled all schools, or all teachers/classrooms within a school. In OLS you can't say anything about teacher effects for unsampled teachers - but for hierarchical regression, the variance component for teachers does allow you to predict what the likely size of non-sampled teacher effects could be.

Additionally, hierarchical regression typically uses "shrinkage", and allows a kind of interpolation between including a particular group of effects in an OLS framework (e.g. teachers), and excluding them. Because of this effect, they are less prone to "overfitting" the data.