Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Also re-order the structure matrix #51

Merged
merged 2 commits into from
Jun 2, 2020

Conversation

keesmulder
Copy link
Contributor

When an oblique rotation has changed the variance order of the factors, the factors are reordered to ensure the first has the greatest variance. However, the structure matrix was assigned using the loadings before reordering, which causes these orders to sometimes be different.

This can cause a lot of surprise for the user; if the factors have been given interpretation, or names, a user may be stumped that the resulting factor scores do not match this interpretation at all, and correlate with different items than the ones with the strongest loadings.

The proposed change is to also reorder the structure matrix, so that the factor scores are always in the same order as the factor loadings. This should theoretically solve the issue everywhere, but the package authors should be careful in whether this change has repercussions elsewhere.

See also:
https://stackoverflow.com/questions/59284044/factor-order-differs-between-loadings-and-scoring-with-oblimin-rotation-how-i

When an oblique rotation has changed the variance order of the factors, the factors are reordered to ensure the first has the greatest variance. However, the structure matrix was assigned using the loadings before reordering, which causes these orders to sometimes be different. 

This can cause a lot of surprise for the user; if the factors have been given interpretation, or names, a user may be stumped that the resulting factor scores do not match this interpretation at all, and correlate with different items than the ones with the strongest loadings. 

The proposed change is to also reorder the structure matrix, so that the factor scores are always in the same order as the factor loadings. This should theoretically solve the issue everywhere, but the package authors should be careful in whether this change has repercussions elsewhere. 

See also: 
https://stackoverflow.com/questions/59284044/factor-order-differs-between-loadings-and-scoring-with-oblimin-rotation-how-i
@keesmulder keesmulder closed this Dec 12, 2019
There was a small error in the PR where sometimes structure is None which can not be reordered, which should be fixed now,
@keesmulder keesmulder reopened this Dec 12, 2019
@codecov-io
Copy link

codecov-io commented Dec 12, 2019

Codecov Report

Merging #51 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #51      +/-   ##
==========================================
+ Coverage   94.51%   94.53%   +0.02%     
==========================================
  Files           5        5              
  Lines         820      823       +3     
==========================================
+ Hits          775      778       +3     
  Misses         45       45
Impacted Files Coverage Δ
factor_analyzer/factor_analyzer.py 94.42% <100%> (+0.07%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a71029f...6c07ffb. Read the comment docs.

@desilinguist
Copy link
Member

@jbiggsets and this one too?

@desilinguist
Copy link
Member

Thank you for this @keesmulder ! Do you happen to have the time to add a test for this change too? If so, that'd be much appreciated!

@desilinguist desilinguist merged commit 9d712e4 into EducationalTestingService:master Jun 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants