Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add statistical test flag on G2 in vocab cooc #383

Open
2 of 7 tasks
Yomguithereal opened this issue Nov 19, 2024 · 0 comments
Open
2 of 7 tasks

Add statistical test flag on G2 in vocab cooc #383

Yomguithereal opened this issue Nov 19, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@Yomguithereal
Copy link
Member

Yomguithereal commented Nov 19, 2024

https://archimede.mat.ulaval.ca/stt1920/STT-1920-Loi-du-khi-deux.pdf

Investigate weird correlation between G2 and chi2 in full version.

I suspect the self-loops are accouting for most of the weirdness.

g2 should return an Option

let observed_12 = (x - xy) as f64;
let observed_21 = (y - xy) as f64;
let observed_22 = (n - (x + y) + xy) as f64;

12 and 21 can be 0

22 can also be 0 and be a problem with self loops?

  • Fix metrics
  • drop --complete, make full computation the default
  • add --approx for non --distribg
  • add test on chi2 for vocab doc-token
  • add test on chi2 or g2 on vocab cooc
  • add lgl specificity on G2
  • avoid self loops altogether in distributional
@Yomguithereal Yomguithereal added the enhancement New feature or request label Nov 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant