Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

update sourmash sig export to export to CSV #1098

Open
ctb opened this issue Jul 13, 2020 · 3 comments
Open

update sourmash sig export to export to CSV #1098

ctb opened this issue Jul 13, 2020 · 3 comments

Comments

@ctb
Copy link
Contributor

ctb commented Jul 13, 2020

https://github.com/dib-lab/2020-ibd/blob/master/scripts/sig_to_csv.py

perhaps do this for import too?
@ctb ctb mentioned this issue Aug 5, 2020
Merged
5 tasks
@ctb ctb mentioned this issue Sep 9, 2020
Closed
@ctb ctb mentioned this issue Jan 1, 2021
Open
@ctb ctb mentioned this issue Jan 16, 2021
Closed
@taylorreiter
Copy link
Contributor

It would be really great if a script like this could be integrated into sourmash sig, with command line parameters to include or remove abundances. Also would be handy to figure out how to deal with multiple kmer sizes -- does ksize become a column where this information is recorded (probably best practice) vs. ksize needing to be selected with a command line param a la compare

@taylorreiter
Copy link
Contributor

I use csvs all the time to R things like make upset plots, run random forests, and generate rarefaction curves

upset plots: #1234 (comment)
upset plots: https://github.com/Arcadia-Science/2022-prjna853785-sourmash/blob/main/notebooks/20220812-visualize-sourmash-signature-intersect.ipynb
rarefaction curves: https://github.com/Arcadia-Science/2022-mtx-not-in-mgx-pairs/blob/ter/specaccum/scripts/calc_rarecurves.R
rarefaction curve viz: https://github.com/Arcadia-Science/2022-mtx-not-in-mgx-pairs/blob/ter/specaccum/notebooks/20220929-visualize-most-converged-rarefaction-curves.ipynb

@taylorreiter taylorreiter mentioned this issue Oct 5, 2022
Closed
5 tasks
@ctb
Copy link
Contributor Author

ctb commented Oct 16, 2022

thought: probably want to upgrade import as well, so that it can take in the same format as export!

@ctb ctb mentioned this issue Dec 1, 2022
Open
@ctb ctb mentioned this issue Jan 3, 2023
Merged
3 tasks
ctb added a commit that referenced this issue Jan 7, 2023
@ctb
[MRG] provide an initial plugin architecture for sourmash that suppor…
14d79c9 
…ts new signature saving & loading mechanisms (#2428)

Implement support for `load_from` and `save_to` plugins via
`importlib.metadata` entry points.

This supports a few of the plugins suggested in
#1353

I am nominating this as an experimental feature that is not under
semantic versioning/not public yet.

Documentation page [here, in
dev_plugins.html](https://sourmash--2428.org.readthedocs.build/en/2428/dev_plugins.html).

A template repo for new plugins is at
https://github.com/sourmash-bio/sourmash_plugin_template.

## Implementation/this PR

This PR refactors the `_load_database` loading and
`SaveSignaturesToLocation` saving code to build a prioritized list of
functions to try in order, and then adds hooks in via the new
`sourmash.plugins` module that insert additional loading/saving
functions into that list.

This PR also moves the current saving/loading functions out of
`sourmash.sourmash_args` into the `sourmash.save_load` submodule, and
simplifies the code a bit.

## Example plugins:

- read JSON sigs and manifests from URLs:
https://github.com/sourmash-bio/sourmash_plugin_load_urls
- read and write signatures in Apache Avro:
https://github.com/sourmash-bio/sourmash_plugin_avro - use extension
`.avrosig` to write.

Specific TODOs:
- [x] provide a minimal "getting started" template repo
- [x] add tests for multiple plugins & priorities
- [ ] maybe try writing CSV export/import as a plugin?
#1098

For later:
- think about other kinds of plugins - new CLI entry points, picklist
classes, tax loading, tax structure, ??.
- work on getting avro support into rust over in
luizirber/2021-02-11-sourmash-binary-format#1
@ctb ctb mentioned this issue Jul 31, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ctb @taylorreiter