Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zero out parcels with <50% coverage #757

Merged
merged 67 commits into from
Jan 31, 2023
Merged

Conversation

tsalo
Copy link
Member

@tsalo tsalo commented Jan 18, 2023

Closes #756.

One thing I'm noticing is that -cifti-correlate has ones in the diagonal, even for parcels with a mean of zero throughout the time series (in numpy these parcels result in a NaN in the diagonal).

Changes proposed in this pull request

  • Measure parcel coverage in both CIFTI and NIFTI parcellated data, and then replace time series from parcels with <50% coverage with zeros.
  • Replace CiftiZerosToNaNs and complicated coverage-measuring steps with CiftiCreateDenseFromTemplate (to map the atlases onto the data file) and CiftiPrepareForParcellation (to handle the parcel coverage issue).
  • Write out a coverage file for each run and each atlas.

@tsalo tsalo added enhancement New feature or request breaking-change PRs that change results or interfaces. labels Jan 18, 2023
@tsalo tsalo self-assigned this Jan 30, 2023
Copy link
Contributor

@mattcieslak mattcieslak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, this was a ton of work

xcp_d/interfaces/prepostcleaning.py Show resolved Hide resolved
)
atlas_file = os.path.join(
connectivity_wf.base_dir,
"connectivity_wf/warp_atlases_to_bold_space/mapflow/_warp_atlases_to_bold_space3",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it safe to look into hardcoded mapnode directories?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's definitely fragile, and any changes to the workflows will inevitably break the tests. The old tests used wildcards to glob through everything, but it was hard to interpret them. I'd love to somehow just look in the workflow object, but AFAICT the inputs and outputs of the different nodes within the workflow can't be accessed like that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if you have access to the workflow object you can use get_node() and then access the input/outputs. Not critical for this PR, but could be a nice addition to the pytest lifestyle

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's AWESOME! I had no idea. I'll definitely clean up the tests using this trick at some point.

xcp_d/tests/test_conn.py Outdated Show resolved Hide resolved
xcp_d/utils/fcon.py Outdated Show resolved Hide resolved
@tsalo tsalo merged commit 497d690 into PennLINC:main Jan 31, 2023
@tsalo tsalo deleted the threshold-coverage branch January 31, 2023 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change PRs that change results or interfaces. enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Zero-out parcels with <50% coverage
3 participants