[讨论]PCA and Rotation in SPSS

是 否 +2 论坛币

k人 参与回答

经管之家送您一份

应届毕业生专属福利!

求职就业群

赵安豆老师微信：zhaoandou666

经管之家联合CDA

送您一个全额奖学金名额~ !

立即领取

感谢您参与论坛问题回答

经管之家送您两个论坛币！

+2 论坛币

Dear Members,

I am new in the list and want to ask a very basic question regarding the principal component analysis. I was doing some analysis by using principal component method and Varimax rotation. However, one of my friends told me that the stat book says that we should not rotate principal components. That is, the principal component analysis should not rotate the solutions, because, by theory, it produces a unique solution. On the other hand, when I read some SPSS manuals, they usually tell you to use the principal component method with some rotation method. Which is correct? Luis O. Benavent

Benavnet Talps Research

Spain

[此贴子已经被作者于2005-9-22 10:47:58编辑过]

扫码加我 拉你入群

请注明：姓名-公司-职位

以便审核进群资格，未注明则拒绝

分享0 收藏0 回帖

关键词：Rotation ATION SPSS ATI TIO 讨论 SPSS pca Rotation

Principal components are a data reduction procedure, not a way to identify interpretable factors. To do the latter, use principal axes analysis or some other factor algorithm that targets common factors. It makes sense to rotate these since you are interested in interpretable factors. However, I suggest an oblique rotation to ensure that the factors are no correlated before forcing them to be. Paul R. Swank, Ph.D. Professor, Developmental Pediatrics Director of Research, Center for Improving the Readiness of Children for Learning and Education (C.I.R.C.L.E.) Medical School UT Health Science Center at Houston

Some clarification:

PCA is designed to maximize the first underlying factor, maximizing its contribution, and (besides) when it extracts all possible factors (number of factor equal to the number of variables) it accounts for the entire variance of the variables. Other methods of extraction do not maximize the first, and besides they try to explain only the COMMON variance, leaving aside a portion of variance that is regarded as unique to each observed variable. Historically, PCA was used first to identify a single factor underlying a set of related measures, supposedly measuring all the same trait (intelligence, as it was), while Principal Axes was used based on the theory that instead of a single Intelligence factor there were underlying "intelligences" for various "faculties of the mind" such as linguistic, graphic or mathematical ability.

In BOTH cases, the position of the coordinate system on which the underlying factors are measured is essentially arbitrary. By rotation, the analyst may be able to put one factor closer to one set of observed variables, and far from others, and the opposite for another factor. This make happen by chance in the initial extraction, but ordinarily doesn't, so rotation is used in order to get that nice characteristic of a factor linked to sets of interrelated variables. For instance, using the same IQ example, by rotation one may get one factor strongly associated with several linguistic tests, and only weakly related to other tests, while another factor is strongly related to mathematical tests and weakly to other tests, so one intuitively calls the first factor "linguistic" and the second "mathematical". These different factors may be independent from each other, or correlated.

It is perfectly possible that being good in language implies being good also in math, and so at least to some degree, some correlation between linguistic and math factors is only to be expected. Initial patterns of extraction ordinarily extract factors that are orthogonal or uncorrelated to each other, because each successive factor is extracted on the unexplained residuals left by the preceding ones, but rotation can position the factor axes in ways that imply they are correlated to each other. Rotation preserving the independence of factors is called orthogonal rotation. Rotation allowing them to be correlated to each other is called oblique rotation. So in Paul Swank response there is a (probably involuntary) confusion at the end: oblique rotation yields correlated factors (though it does not FORCE them to be correlated) while orthogonal rotation methods FORCE rotated factors to be uncorrelated to each other.

Hector

Hi Paul and Hector, Thank you very much for your explanations and clarifications. I sincerely appreciate your help.

I want to ask one more question to Hector. In your explanations, you seem to be saying that in BOTH cases (i.e. PCA and principal axes method) we can rotate the principal components in order to "put one factor closer to one set of observed variables, and far from others, and the opposite for another factor". Is this a correct procedure in terms of PCA? I am asking this question, because this is precisely the doubt I have. Some of my stat friends contend that in PCA we should not rotate principal components, because it is a data reduction method (as Paul explains), and produces a best linear combination of the variance. They even say that if we rotate the principal components, they are no longer "principal" components. In short, their point is that PCA and FA are two different procedures, and the rotation should be applied ONLY in the case of FA. I would sincerely appreciate your further clarifications.

Luis

Hi, Hector,

Thank you for your clear explanations. Your comments bring me to the ultimate question: when we use SPSS's "Factor Analysis" with the "Principal Component" method and VARIMAX rotation. Are we conducting FA or PCA? I realized that some researchers are arguing that SPSS's "Factor Analysis" with the default function of "Principal component" is causing a great confusion in distinguishing between FA and PCA. Because of this, my stat friends recommend not to choose principal component method in FA. Furthermore, their explanations are accidentally similar to what Paul suggests: the use of oblique rotation.

Luis

Hi, Luis,

Do not get confused by mere names. Factor analysis includes Principal Component Analysis as one of its methods for factor extraction. In fact, PCA and FA are synonymous, but historically the introduction of factor analysis was done using PCA as a method for factor extraction, and thence the different denominations that still persist in textbooks. SPSS Factor procedure involves both. The procedure could be called PCA, and that would amount to the same. Besides, doing an oblique or orthogonal rotation is a different question altogether. If you rotate the factors or components in an oblique faction, you end up with a number of underlying or latent variables (the "factors") that are correlated to each other. Therefore, part of the variance in the original variables will be shared between two or more of these factors. Moreover, once you obtain an oblique solution, you could still perform a second-order factor analysis to find the common underlying factor/s that explain the correlations between your first-order underlying factors. (I hope you do not make the second-order factors correlated too, for this would suggest a third-order factorization and so on until you run out of degrees of freedom --or patience).

Independent or orthogonal factors explain parts of the variance in the variables that do not overlap. After letting Factor 1 explain as much as it can, Factor 2 explains as much as it can of the residuals, and so on. So Factors 1 and 2 may be regarded as completely unrelated variables. Correlated factors (those resulting from an oblique rotation) cannot be identified with different unrelated variables; since they share part of their variability with each other, they may be seen in part as the expression of some common, deeper variable. If you want this game to end, you need to arrive either to one overarching factor that explains most of the variance, or to several UNRELATED factors each explaining a part of it. The classical example of intelligence is useful in this context. The initial work by Spearman used PCA to identify a "general intelligence" factor, correlated with all intelligence tests and even with most of the individual items in those tests. The rest of the variance was attributed to random error and to unique factors associated with each individual test. Years later Thurstone developed the idea of multiple intelligences (linguistic, visual, logical, etc.), and used Principal Axes Factoring to extract several independent factors, later rotated in orthogonal or oblique fashion to make them pass through the appropriate tests (i.e. the "linguistic" factor was a factor that could be made, through rotation, to have high loadings on linguistic tests and low loadings on other tests). Since the same data set could be analyzed either way, yielding one general or several partial intelligence factors, analysts have long recognized that "factors" are mathematical constructs and not real objects. One uses them to summarize data and illustrate theory, and may choose one or another depending on external evidence, better data fit, or other considerations.

Hector

I highly recommend reading:

Preacher, K. J., & MacCallum, R. C. (2003). Repairing Tom Swift's electric factor analysis machine. Understanding Statistics, 2(1), 13-32.

It describes the "little jiffy", principal components analysis followed by a varimax rotation, as the most commonly used, yet incorrect, factoring procedure. It is incorrect because PCA will typically underfactor, especially if using Kaiser’s eigenvalue > 1 criterion, it does not attempt to eliminate specific variance, and most of the time, factors will have some correlation.

Paul R. Swank, Ph.D. Professor, Developmental Pediatrics Director of Research, Center for Improving the Readiness of Children for Learning and Education (C.I.R.C.L.E.) Medical School UT Health Science Center at Houston

I have not read Preacher and MacCallum's article. However, some remarks:

• From n variables one can extract up to n factors. If one does, and uses PCA, one explains 100% of total variance in the n variables. With other methods, n factors explain 100% of the commonality. If one extracts only some of the factors, one explains only part of total variance or total commonality.
• There is no objective criterion to stop extracting factors. One can stop after one factor, two, three, 14 or 28 with equal theoretical justification. One can aim at explaining 20% of total variance, or 50%, or 90%, depending on goals and circumstances. Therefore there is no under- or over-factoring as such, only relative to some standard of comparison. And the decision to extract fewer or more numerous factors can be made with any of the factoring methods, not just with PCA.
• One criterion for determining the number of factors is stopping at the last factor whose eigenvalue is >1, on the fragile grounds that factors with eigenvalues >1 explain more variance than any observed variable (whose "eigenvalue" is 1 by convention). This criterion is essentially arbitrary, and has been criticized many times. For different purposes you may want fewer or more factors.
• Another criterion is the so-called scree curve: stop when the eigenvalue of a new factor differs only little from the preceding one (i.e. when the slope of the eigenvalues curve becomes flatter). This is still less justifiable than the above, but still widely used.
• In practical work one often needs (or has theoretical grounds to seek) only the main one or just a few factors, and the extraction stops when the analyst seems fit. In some specific applications one is trying to capture as much as possible of total variance (or total commonality as the case might be), and so one wants to use more factors. Since factors are just shorthand constructs for the intercorrelation of variables, it is all in the hands of the analyst, and one may take any of these decisions without fear of divine retribution.
  
  

Hector

[此贴子已经被作者于2005-9-22 10:59:35编辑过]

PCA does not distinguish between common and specific factor variance and does not explicity model the error variance. In studies where the factor structure is known, PCA with the eigenvalue > 1 rule will often incorrectly identify the number of factors present. Thirdly, rotation is used for interpretation of the factors. Why rotate if not to interpret the factors? Or components in the case of PCA. But since the components are not common factors since they can contain specific variance, why try to interpret them. If one wants to examine the underlying structure of the data to find the dimensions represented, then one should look at common factors with principal axes or maximum likelihood methods. Finally, in the areas of biomedical and social sciences in which I work, there are rarely uncorrelated factors. In fact there is a saying that all psychological variables are correlated at least .30. This of course is a rank generalization but it is true that many variables, and therefore factors, are correlated. If the factors are not correlated then the oblique solution will be similar to the orthogonal one so what is lost by doing the oblique solution in the first place?

Considering the variance accounted for by the components is a characteristic of data reduction, not interpretation. If all I want to do is account for the maximum variance with the smallest number of indepedent components, then PCA with variamax rotation is okay. But if one is attempting to understand the underlying dimensions then the percentage of variance accounted for is less applicable. Of more interest is accounting for all of the common factor variance. Then one can eliminate factors that do not make sense or are otherwise not interpreatble.


Paul R. Swank, Ph.D.
Professor, Developmental Pediatrics
Director of Research, Center for Improving the Readiness of Children for Learning and Education (C.I.R.C.L.E.)
Medical School
UT Health Science Center at Houston