Ingest sequencing accession IDs #387
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adding a function to clinical ETL module to parse sequencing accession ID tracking sheets that contain NWGC IDs, BBI-assigned strain names, GISAID, and GenBank accessions. This function parses, filters, and formats the data for ingestion into ID3C.
…RSV tracking sheet format.
Calculates a sequence identifier by hashing the `strain_name` and appending the pathogen code (RSVA, RSVB, or HCOV19) to be used as the identifier in warehouse.genomic_sequence table.
…eceiving table The clinical ETL is being updated for ingestion of sequencing accessioning data from receiving.clinical. This data is being sourced from tracking sheets maintained on Github for the Seattle Flu Study. Running this ETL on receiving.clinical records with document containing `gisaid_accession` or `genbank_accession` will result in custom processing for this particular type of data. After matching to an existing sample, a minimal `consensus_genome` and `genomic_sequence` record will be generated for each covid-19, RSV-A, and RSV-B sequence record.
davereinhart
force-pushed
the
ingest-seq-accession-ids
branch
from
25c298e to
3929359
Compare
Updating ETL to process sequencing accession identifier data from SFS from the clinical receiving table. This data is being processed through the clinical ETL because it has a distinct format from previous sequencing data, and is likely to only be needed once at project close.
The upsert_genome function was only performing inserts. Adding ON CONFLICT clause to perform a "non-updating" update and return the existing record id.
davereinhart
force-pushed
the
ingest-seq-accession-ids
branch
from
3929359 to
e306ec1
Compare
jstone-dev
approved these changes
Oct 2, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The clinical data module is being updated to parse sequencing accessioning data for uploading into receiving.clinical, and the clinical ETL is being updated for ingestion into warehouse tables. This data is being sourced from tracking sheets maintained on Github for the Seattle Flu Study.
Running this ETL on receiving.clinical records with document containing
gisaid_accessionorgenbank_accessionwill result in custom processing for this particular type of data. After matching to an existing sample, a minimalconsensus_genomeandgenomic_sequencerecord will be generated for each covid-19, RSV-A, RSV-B, Influenza A and Influenza B sequence record.