-
Notifications
You must be signed in to change notification settings - Fork 81
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Web API Endpoint to programmatically upload biom for a study #2320
Comments
Sorry, fat fingers. I think the main problem is that to be able to add a biom we first need a study with a sample info file and prep info file; you can add a biom to a prep. So perhaps adding we can allow study, sample and prep info creation via the API or some way figure out how to transfer that info back and forth. Now, this doesn't actually solve the issue with GNPS as it doesn't have sample information (not sure about prep) so I think if we add the raw data to Qiita, then send the info for processing to GNPS, then send back the biom, will be the best way forward. @mwang87, what do you think? |
@antgonza, I think those points are covered by the REST API endpoints that @wasade and I put together a few months ago. See here and here for the full list of endpoints. The main problem is that these are not exposed to users outside of Qiita, exposing this might take some work. |
Right, thanks. We could start with only internal and go from there ... |
Cool, that makes sense. Are internal APIs just not exposed by blocking ports or could we just have a credentialing process? |
I just want to drop my 2 cents here. As per previous discussions with @mwang87 and others we agreed that creating a plugin would be the best way of connecting the 2 systems. Following this route, no new development in Qiita should be done as the plugin infrastructure should be complete. |
The process to create credentials for a given user is not exposed to the
public, so we would have to figure out something on a case-by-case
basis, which I'm sure we can do :)
…On (Oct-03-17|18:16), Ming Wang wrote:
Cool, that makes sense. Are internal APIs just not exposed by blocking ports or could we just have a credentialing process?
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
#2320 (comment)
|
^ Right, that's a more sustainable alternative.
…On (Oct-03-17|18:19), Jose Navas wrote:
I just want to drop my 2 cents here. As per previous discussions with @mwang87 and others we agreed that creating a plugin would be the best way of connecting the 2 systems. Following this route, no new development in Qiita should be done as the plugin infrastructure should be complete.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
#2320 (comment)
|
@josenavas Yes, I think it would be great if users are starting at Qiita with also sequencing data. I do also think there are a lot of people that come from the mass spectrometry side that would like to have access to the statistical tools available at Qiita that are not familiar with the platform but are familiar with GNPS. Currently, we automatically generate a subset of emperor features, but I want to avoid recreating all the analysis features that are already implemented in Qiita. |
plugin will be the best. Figuring out how to "request" sample/prep info from GNPS users, will be the best. As you know, without metadata analysis are impossible. BTW we have this: https://github.com/qiita-spots/qtp-mass-spec |
* fix #2320 * adding prints to debug * children -> 1 * APIArtifactHandlerTest -> APIArtifactHandlerTests * configure_biom * qdb.util.activate_or_update_plugins * improving code * almost there * add values.template * fix filepaths * filepaths -> files * fixing errors * add prep.artifact insertion * addressing @ElDeveloper comments
* Version 092020 (#3034) * inject study_type in EBI and improvements to current automatic processing pipeline (#3023) * inject study_type in ebi and improvements to current automatic proecssing pipeline * addressing @ElDeveloper comments * some general fixes/additions for next release (#3026) * some general fixes/additions for next release * adding test for not None job.release_validator_job * fix #2839 * fix #2868 (#3028) * fix #2868 * 2nd round * fix errors * more changes * fix errors * fix ProcessingJobTest * fix PY_PATCH * add missing TRN.add * encapsulated_query -> perform_as_transaction * fix #3022 (#3030) * fix #3022 * adding tests * fix #2320 (#3031) * fix #2320 * adding prints to debug * children -> 1 * APIArtifactHandlerTest -> APIArtifactHandlerTests * configure_biom * qdb.util.activate_or_update_plugins * improving code * almost there * add values.template * fix filepaths * filepaths -> files * fixing errors * add prep.artifact insertion * addressing @ElDeveloper comments * fix artifact_definition active command * != -> == * Added three tutorial sections to the Qiita documentation (#3032) * Added three tutorial sections to the Qiita documentation: 'Retrieving Public Data for Own Analysis' and 'Processing public data retrieved with redbiom' to the redbiom tab, and 'Statistical Analysis to Justify Clinical Trial Sample Size Tutorial' to the analyzing samples tab. * Update redbiom.rst * Update redbiom.rst * Update redbiom.rst * Further updates to redbiom.rst and the Stats tutorial. * update redbiom.rst * Finished proof-reading * Placed all three tutorials/sections together under Introduction to the download and analysis of public Qiita data * added a new introduction, with links to the three sections * Added figures to stats tutorial and contexts explanation * Added figures to stats tutorial and contexts explanation * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Antonio Gonzalez <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> * 092020 (#3033) * 092020 * connect artifact with job * rm INSERT qiita.artifact_processing_job * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Daniel McDonald <[email protected]> Co-authored-by: Mirte Kuijpers <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> * fix #3036 Co-authored-by: Daniel McDonald <[email protected]> Co-authored-by: Mirte Kuijpers <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]>
…3040) * Version 092020 (#3034) * inject study_type in EBI and improvements to current automatic processing pipeline (#3023) * inject study_type in ebi and improvements to current automatic proecssing pipeline * addressing @ElDeveloper comments * some general fixes/additions for next release (#3026) * some general fixes/additions for next release * adding test for not None job.release_validator_job * fix #2839 * fix #2868 (#3028) * fix #2868 * 2nd round * fix errors * more changes * fix errors * fix ProcessingJobTest * fix PY_PATCH * add missing TRN.add * encapsulated_query -> perform_as_transaction * fix #3022 (#3030) * fix #3022 * adding tests * fix #2320 (#3031) * fix #2320 * adding prints to debug * children -> 1 * APIArtifactHandlerTest -> APIArtifactHandlerTests * configure_biom * qdb.util.activate_or_update_plugins * improving code * almost there * add values.template * fix filepaths * filepaths -> files * fixing errors * add prep.artifact insertion * addressing @ElDeveloper comments * fix artifact_definition active command * != -> == * Added three tutorial sections to the Qiita documentation (#3032) * Added three tutorial sections to the Qiita documentation: 'Retrieving Public Data for Own Analysis' and 'Processing public data retrieved with redbiom' to the redbiom tab, and 'Statistical Analysis to Justify Clinical Trial Sample Size Tutorial' to the analyzing samples tab. * Update redbiom.rst * Update redbiom.rst * Update redbiom.rst * Further updates to redbiom.rst and the Stats tutorial. * update redbiom.rst * Finished proof-reading * Placed all three tutorials/sections together under Introduction to the download and analysis of public Qiita data * added a new introduction, with links to the three sections * Added figures to stats tutorial and contexts explanation * Added figures to stats tutorial and contexts explanation * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Antonio Gonzalez <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> * 092020 (#3033) * 092020 * connect artifact with job * rm INSERT qiita.artifact_processing_job * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Daniel McDonald <[email protected]> Co-authored-by: Mirte Kuijpers <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> * adding job submit ENVIRONMENT and minor improvement to runWorkflow * fix test Co-authored-by: Daniel McDonald <[email protected]> Co-authored-by: Mirte Kuijpers <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]>
* Version 092020 (#3034) * inject study_type in EBI and improvements to current automatic processing pipeline (#3023) * inject study_type in ebi and improvements to current automatic proecssing pipeline * addressing @ElDeveloper comments * some general fixes/additions for next release (#3026) * some general fixes/additions for next release * adding test for not None job.release_validator_job * fix #2839 * fix #2868 (#3028) * fix #2868 * 2nd round * fix errors * more changes * fix errors * fix ProcessingJobTest * fix PY_PATCH * add missing TRN.add * encapsulated_query -> perform_as_transaction * fix #3022 (#3030) * fix #3022 * adding tests * fix #2320 (#3031) * fix #2320 * adding prints to debug * children -> 1 * APIArtifactHandlerTest -> APIArtifactHandlerTests * configure_biom * qdb.util.activate_or_update_plugins * improving code * almost there * add values.template * fix filepaths * filepaths -> files * fixing errors * add prep.artifact insertion * addressing @ElDeveloper comments * fix artifact_definition active command * != -> == * Added three tutorial sections to the Qiita documentation (#3032) * Added three tutorial sections to the Qiita documentation: 'Retrieving Public Data for Own Analysis' and 'Processing public data retrieved with redbiom' to the redbiom tab, and 'Statistical Analysis to Justify Clinical Trial Sample Size Tutorial' to the analyzing samples tab. * Update redbiom.rst * Update redbiom.rst * Update redbiom.rst * Further updates to redbiom.rst and the Stats tutorial. * update redbiom.rst * Finished proof-reading * Placed all three tutorials/sections together under Introduction to the download and analysis of public Qiita data * added a new introduction, with links to the three sections * Added figures to stats tutorial and contexts explanation * Added figures to stats tutorial and contexts explanation * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Antonio Gonzalez <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> * 092020 (#3033) * 092020 * connect artifact with job * rm INSERT qiita.artifact_processing_job * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Daniel McDonald <[email protected]> Co-authored-by: Mirte Kuijpers <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> * fix #2920 * fix test Co-authored-by: Daniel McDonald <[email protected]> Co-authored-by: Mirte Kuijpers <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]>
* Version 092020 (#3034) * inject study_type in EBI and improvements to current automatic processing pipeline (#3023) * inject study_type in ebi and improvements to current automatic proecssing pipeline * addressing @ElDeveloper comments * some general fixes/additions for next release (#3026) * some general fixes/additions for next release * adding test for not None job.release_validator_job * fix #2839 * fix #2868 (#3028) * fix #2868 * 2nd round * fix errors * more changes * fix errors * fix ProcessingJobTest * fix PY_PATCH * add missing TRN.add * encapsulated_query -> perform_as_transaction * fix #3022 (#3030) * fix #3022 * adding tests * fix #2320 (#3031) * fix #2320 * adding prints to debug * children -> 1 * APIArtifactHandlerTest -> APIArtifactHandlerTests * configure_biom * qdb.util.activate_or_update_plugins * improving code * almost there * add values.template * fix filepaths * filepaths -> files * fixing errors * add prep.artifact insertion * addressing @ElDeveloper comments * fix artifact_definition active command * != -> == * Added three tutorial sections to the Qiita documentation (#3032) * Added three tutorial sections to the Qiita documentation: 'Retrieving Public Data for Own Analysis' and 'Processing public data retrieved with redbiom' to the redbiom tab, and 'Statistical Analysis to Justify Clinical Trial Sample Size Tutorial' to the analyzing samples tab. * Update redbiom.rst * Update redbiom.rst * Update redbiom.rst * Further updates to redbiom.rst and the Stats tutorial. * update redbiom.rst * Finished proof-reading * Placed all three tutorials/sections together under Introduction to the download and analysis of public Qiita data * added a new introduction, with links to the three sections * Added figures to stats tutorial and contexts explanation * Added figures to stats tutorial and contexts explanation * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Antonio Gonzalez <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> * 092020 (#3033) * 092020 * connect artifact with job * rm INSERT qiita.artifact_processing_job * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Daniel McDonald <[email protected]> Co-authored-by: Mirte Kuijpers <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> * rm create_qiime_mapping_file * fixing some tests * fixing more tests * fix even more tests * rm npt.assert_warns * qiime-map -> sample-file * update not_merged_samples.txt * adding @ElDeveloper changes Co-authored-by: Daniel McDonald <[email protected]> Co-authored-by: Mirte Kuijpers <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]>
* Version 092020 (#3034) * inject study_type in EBI and improvements to current automatic processing pipeline (#3023) * inject study_type in ebi and improvements to current automatic proecssing pipeline * addressing @ElDeveloper comments * some general fixes/additions for next release (#3026) * some general fixes/additions for next release * adding test for not None job.release_validator_job * fix #2839 * fix #2868 (#3028) * fix #2868 * 2nd round * fix errors * more changes * fix errors * fix ProcessingJobTest * fix PY_PATCH * add missing TRN.add * encapsulated_query -> perform_as_transaction * fix #3022 (#3030) * fix #3022 * adding tests * fix #2320 (#3031) * fix #2320 * adding prints to debug * children -> 1 * APIArtifactHandlerTest -> APIArtifactHandlerTests * configure_biom * qdb.util.activate_or_update_plugins * improving code * almost there * add values.template * fix filepaths * filepaths -> files * fixing errors * add prep.artifact insertion * addressing @ElDeveloper comments * fix artifact_definition active command * != -> == * Added three tutorial sections to the Qiita documentation (#3032) * Added three tutorial sections to the Qiita documentation: 'Retrieving Public Data for Own Analysis' and 'Processing public data retrieved with redbiom' to the redbiom tab, and 'Statistical Analysis to Justify Clinical Trial Sample Size Tutorial' to the analyzing samples tab. * Update redbiom.rst * Update redbiom.rst * Update redbiom.rst * Further updates to redbiom.rst and the Stats tutorial. * update redbiom.rst * Finished proof-reading * Placed all three tutorials/sections together under Introduction to the download and analysis of public Qiita data * added a new introduction, with links to the three sections * Added figures to stats tutorial and contexts explanation * Added figures to stats tutorial and contexts explanation * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Antonio Gonzalez <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> * 092020 (#3033) * 092020 * connect artifact with job * rm INSERT qiita.artifact_processing_job * Apply suggestions from code review [skip ci] Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Daniel McDonald <[email protected]> Co-authored-by: Mirte Kuijpers <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> * adding qiita.study autoloaded column * cleaning tests * fix redbiom test * avoiding clog submissions * Apply suggestions from code review Co-authored-by: Yoshiki Vázquez Baeza <[email protected]> Co-authored-by: Daniel McDonald <[email protected]> Co-authored-by: Mirte Kuijpers <[email protected]> Co-authored-by: Yoshiki Vázquez Baeza <[email protected]>
Web API to upload a biom file (or provide a URL for qiita to pull down the file) would be useful for a service like GNPS to export biom file for metabolomics analysis to be analyzed easily in Qiita directly and easily for GNPS users.
The text was updated successfully, but these errors were encountered: