From 3eaeabdc9522fc09ebbba36d8080c992bd283544 Mon Sep 17 00:00:00 2001 From: Cristina Yenyxe Gonzalez Garcia Date: Thu, 7 Jun 2018 15:29:43 +0100 Subject: [PATCH 1/8] Added class to each URL in the response --- htsget.md | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/htsget.md b/htsget.md index 90677a17d..5995ec611 100644 --- a/htsget.md +++ b/htsget.md @@ -280,6 +280,12 @@ _optional object_ For HTTPS URLs, the server may supply a JSON object containing one or more string key-value pairs which the client MUST supply as headers with any request to the URL. For example, if headers is `{"Range": "bytes=0-1023", "Authorization": "Bearer xxxx"}`, then the client must supply the headers `Range: bytes=0-1023` and `Authorization: Bearer xxxx` with the HTTPS request to the URL. + +`class` +_string_ + +For file formats whose specification describes a header and a body, the class indicates which of the two will be retrieved when querying this URL. The allowed values are `header` and `body`. + @@ -300,24 +306,28 @@ An example of a JSON response is: "format" : "BAM", "urls" : [ { - "url" : "data:application/vnd.ga4gh.bam;base64,QkFNAQ==" + "url" : "data:application/vnd.ga4gh.bam;base64,QkFNAQ==", + "class" : "header" }, { - "url" : "https://htsget.blocksrv.example/sample1234/header" + "url" : "https://htsget.blocksrv.example/sample1234/header", + "class" : "header" }, { "url" : "https://htsget.blocksrv.example/sample1234/run1.bam", "headers" : { "Authorization" : "Bearer xxxx", "Range" : "bytes=65536-1003750" - } + }, + "class" : "body" }, { "url" : "https://htsget.blocksrv.example/sample1234/run1.bam", "headers" : { "Authorization" : "Bearer xxxx", "Range" : "bytes=2744831-9375732" - } + }, + "class" : "body" } ] } From 8ae6f0123cd5276504167643d78ef2dd40e4cc89 Mon Sep 17 00:00:00 2001 From: Cristina Yenyxe Gonzalez Garcia Date: Fri, 7 Sep 2018 15:09:54 +0100 Subject: [PATCH 2/8] All or none URL class in the response --- htsget.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/htsget.md b/htsget.md index 5995ec611..2e60b9a11 100644 --- a/htsget.md +++ b/htsget.md @@ -284,7 +284,7 @@ For HTTPS URLs, the server may supply a JSON object containing one or more strin `class` _string_ -For file formats whose specification describes a header and a body, the class indicates which of the two will be retrieved when querying this URL. The allowed values are `header` and `body`. +For file formats whose specification describes a header and a body, the class indicates which of the two will be retrieved when querying this URL. Either all or none of the URLs in the response must have a class attribute. The allowed values are `header` and `body`. From 1ece917620d0a0179327e184e94cf1b9d8b0cce4 Mon Sep 17 00:00:00 2001 From: Cristina Yenyxe Gonzalez Garcia Date: Fri, 7 Sep 2018 15:22:52 +0100 Subject: [PATCH 3/8] URL class as request parameter --- htsget.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/htsget.md b/htsget.md index 2e60b9a11..44b8db68a 100644 --- a/htsget.md +++ b/htsget.md @@ -162,6 +162,13 @@ The server SHOULD reply with an `UnsupportedFormat` error if the requested forma [^a] +`class` +_optional string_ + +A list of URL classes to include, see below. +Default: all + + `referenceName` _optional_ From c9968e90edeb242fb602402c62cae5c6088ed8e0 Mon Sep 17 00:00:00 2001 From: Cristina Yenyxe Gonzalez Garcia Date: Thu, 29 Nov 2018 11:01:34 +0000 Subject: [PATCH 4/8] Clarification about class attribute in response --- htsget.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/htsget.md b/htsget.md index 44b8db68a..b9cee693e 100644 --- a/htsget.md +++ b/htsget.md @@ -291,7 +291,9 @@ For HTTPS URLs, the server may supply a JSON object containing one or more strin `class` _string_ -For file formats whose specification describes a header and a body, the class indicates which of the two will be retrieved when querying this URL. Either all or none of the URLs in the response must have a class attribute. The allowed values are `header` and `body`. +For file formats whose specification describes a header and a body, the class indicates which of the two will be retrieved when querying this URL. The allowed values are `header` and `body`. + +Either all or none of the URLs in the response must have a class attribute. If it is absent, clients should assume data blocks include both header and body, possibly mixed in one of the blocks. From 82791f1e4575c12f160f230480c01be49d83b7b9 Mon Sep 17 00:00:00 2001 From: Cristina Yenyxe Gonzalez Garcia Date: Thu, 29 Nov 2018 11:11:25 +0000 Subject: [PATCH 5/8] Capitalization according to RFC 2119 --- htsget.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/htsget.md b/htsget.md index b9cee693e..9518ec10c 100644 --- a/htsget.md +++ b/htsget.md @@ -293,7 +293,7 @@ _string_ For file formats whose specification describes a header and a body, the class indicates which of the two will be retrieved when querying this URL. The allowed values are `header` and `body`. -Either all or none of the URLs in the response must have a class attribute. If it is absent, clients should assume data blocks include both header and body, possibly mixed in one of the blocks. +Either all or none of the URLs in the response MUST have a class attribute. If it is absent, clients should assume data blocks include both header and body, possibly mixed in one of the blocks. From 677e158a054b569ecc3bde822e9005ca90291440 Mon Sep 17 00:00:00 2001 From: Cristina Yenyxe Gonzalez Garcia Date: Thu, 29 Nov 2018 11:19:49 +0000 Subject: [PATCH 6/8] Description of 'class' query parameter --- htsget.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/htsget.md b/htsget.md index 9518ec10c..38ebdaf68 100644 --- a/htsget.md +++ b/htsget.md @@ -165,7 +165,7 @@ The server SHOULD reply with an `UnsupportedFormat` error if the requested forma `class` _optional string_ -A list of URL classes to include, see below. +Allows clients to request the data header only, using `class=header`. It is not possible to return the data body only, because that would not constitute a valid response in any of the supported formats. Default: all From 7ecff43351f6734360b3a918da65b84923ea678c Mon Sep 17 00:00:00 2001 From: John Marshall Date: Thu, 24 Jan 2019 09:42:25 +0000 Subject: [PATCH 7/8] Add class=header request text and "core mechanic" explanation of classes Describe the class=header request query parameter more fully. Add description of using response URL classes to the "diagram of core mechanic" section. [SQUASH INTO PREVIOUS] Add "optional", some wordsmithing, and add blank lines so formatting works when displayed in the GH repo too.] --- htsget.md | 31 ++++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/htsget.md b/htsget.md index 38ebdaf68..b0c83a250 100644 --- a/htsget.md +++ b/htsget.md @@ -162,11 +162,26 @@ The server SHOULD reply with an `UnsupportedFormat` error if the requested forma [^a] + `class` _optional string_ -Allows clients to request the data header only, using `class=header`. It is not possible to return the data body only, because that would not constitute a valid response in any of the supported formats. -Default: all + +Request different classes of data. +By default, i.e., when `class` is not specified, the response will represent a complete read or variant data stream, encompassing SAM/CRAM/VCF headers, body data records, and EOF marker. + +If `class` is specified, its value MUST be one of the following: + + +
+ +`header` + + +Request the SAM/CRAM/VCF headers only. + +The server SHOULD respond with an `InvalidInput` error if any other htsget query parameters other than `format` are specified at the same time as `class=header`. +
`referenceName` @@ -288,12 +303,15 @@ _optional object_ For HTTPS URLs, the server may supply a JSON object containing one or more string key-value pairs which the client MUST supply as headers with any request to the URL. For example, if headers is `{"Range": "bytes=0-1023", "Authorization": "Bearer xxxx"}`, then the client must supply the headers `Range: bytes=0-1023` and `Authorization: Bearer xxxx` with the HTTPS request to the URL. + `class` -_string_ +_optional string_ + For file formats whose specification describes a header and a body, the class indicates which of the two will be retrieved when querying this URL. The allowed values are `header` and `body`. -Either all or none of the URLs in the response MUST have a class attribute. If it is absent, clients should assume data blocks include both header and body, possibly mixed in one of the blocks. +Either all or none of the URLs in the response MUST have a class attribute. +If `class` fields are not supplied, no assumptions can be made about which data blocks contain headers, body records, or parts of both. @@ -354,7 +372,10 @@ An example of a JSON response is: 3. Client fetches the data blocks using the URLs and headers. 4. Client concatenates data blocks to produce local blob. -While the blocks must be finally concatenated in the given order, the client may fetch them in parallel. +While the blocks must be finally concatenated in the given order, the client may fetch them in parallel and/or reuse cached data from URLs that have previously been downloaded. + +When making a series of requests to fetch reads or variants within different regions of the same `` resource, clients may wish to avoid re-fetching the SAM/CRAM/VCF headers each time, especially if they are large. +If the ticket contains `class` fields, the client may reuse previously downloaded and parsed headers rather than re-fetching the `header`-class URLs. ### HTTPS data block URLs From d13b103103411548441a67f386f917d23f275cf6 Mon Sep 17 00:00:00 2001 From: Mike Lin Date: Mon, 22 Apr 2019 11:27:14 -0700 Subject: [PATCH 8/8] revert my merge error --- htsget.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/htsget.md b/htsget.md index dfb09650a..daf5ccb77 100644 --- a/htsget.md +++ b/htsget.md @@ -188,6 +188,8 @@ Request the SAM/CRAM/VCF headers only. The server SHOULD respond with an `InvalidInput` error if any other htsget query parameters other than `format` are specified at the same time as `class=header`. + + `referenceName` _optional_