Variable capture not working

[ad8lmondy]

Hello,

This is a docs issue as well as a code issue (I think!).

If I try the example under https://hurl.dev/docs/capturing-response.html#variable-capture :

GET https://example.org/helloworld

HTTP/1.0 200
[Captures]
in: body
name: variable "in" regex "Hello ([a-zA-Z]+)!"

It reports:

hurl --very-verbose --no-output sample.hurl 
* Options:
*     fail fast: true
*     follow redirect: false
*     insecure: false
*     max redirect: 50
*     retry: false
*     retry max count: 10
* ------------------------------------------------------------------------------
* Executing entry 1
*
* Cookie store:
*
* Request:
* GET https://example.org/helloworld
*
* Request can be run with the following curl command:
* curl 'https://example.org/helloworld'
*
**   Trying 93.184.216.34:443...
**   Trying 2606:2800:220:1:248:1893:25c8:1946:443...
** Immediate connect fail for 2606:2800:220:1:248:1893:25c8:1946: Network is unreachable
** Connected to example.org (93.184.216.34) port 443 (#0)
** ALPN, offering h2
** ALPN, offering http/1.1
**  CAfile: /etc/ssl/certs/ca-certificates.crt
**  CApath: /etc/ssl/certs
** TLSv1.0 (OUT), TLS header, Certificate Status (22):
** TLSv1.3 (OUT), TLS handshake, Client hello (1):
** TLSv1.2 (IN), TLS header, Certificate Status (22):
** TLSv1.3 (IN), TLS handshake, Server hello (2):
** TLSv1.2 (OUT), TLS header, Finished (20):
** TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
** TLSv1.2 (OUT), TLS header, Certificate Status (22):
** TLSv1.3 (OUT), TLS handshake, Client hello (1):
** TLSv1.2 (IN), TLS header, Finished (20):
** TLSv1.2 (IN), TLS header, Certificate Status (22):
** TLSv1.3 (IN), TLS handshake, Server hello (2):
** TLSv1.2 (IN), TLS header, Supplemental data (23):
** TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
** TLSv1.2 (IN), TLS header, Supplemental data (23):
** TLSv1.3 (IN), TLS handshake, Certificate (11):
** TLSv1.2 (IN), TLS header, Supplemental data (23):
** TLSv1.3 (IN), TLS handshake, CERT verify (15):
** TLSv1.2 (IN), TLS header, Supplemental data (23):
** TLSv1.3 (IN), TLS handshake, Finished (20):
** TLSv1.2 (OUT), TLS header, Supplemental data (23):
** TLSv1.3 (OUT), TLS handshake, Finished (20):
** SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
** ALPN, server accepted to use h2
** Server certificate:
**  start date: Mar 14 00:00:00 2022 GMT
**  expire date: Mar 14 23:59:59 2023 GMT
**  subjectAltName: host "example.org" matched cert's "example.org"
**  issuer: C=US; O=DigiCert Inc; CN=DigiCert TLS RSA SHA256 2020 CA1
**  SSL certificate verify ok.
** Using HTTP2, server supports multiplexing
** Connection state changed (HTTP/2 confirmed)
** Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
** TLSv1.2 (OUT), TLS header, Supplemental data (23):
** TLSv1.2 (OUT), TLS header, Supplemental data (23):
** TLSv1.2 (OUT), TLS header, Supplemental data (23):
** Using Stream ID: 1 (easy handle 0x5606e2778f50)
** TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /helloworld HTTP/2
> Host: example.org
> accept: */*
> user-agent: hurl/1.8.0
>
* Request body:
*
** TLSv1.2 (IN), TLS header, Supplemental data (23):
** TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
** TLSv1.2 (IN), TLS header, Supplemental data (23):
** TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
** old SSL session ID is stale, removing
** TLSv1.2 (IN), TLS header, Supplemental data (23):
** TLSv1.2 (OUT), TLS header, Supplemental data (23):
** TLSv1.2 (IN), TLS header, Supplemental data (23):
** TLSv1.2 (IN), TLS header, Supplemental data (23):
** TLSv1.2 (IN), TLS header, Supplemental data (23):
** TLSv1.2 (IN), TLS header, Supplemental data (23):
** TLSv1.2 (IN), TLS header, Supplemental data (23):
** Connection #0 to host example.org left intact
* Response: (received 1256 bytes in 1057 ms)
*
< HTTP/2 404
< accept-ranges: bytes
< age: 41949
< cache-control: max-age=604800
< content-type: text/html; charset=UTF-8
< date: Tue, 29 Nov 2022 01:25:00 GMT
< expires: Tue, 06 Dec 2022 01:25:00 GMT
< last-modified: Mon, 28 Nov 2022 13:45:51 GMT
< server: ECS (sab/56F1)
< vary: Accept-Encoding
< x-cache: 404-HIT
< content-length: 1256
<
* Response body:
* <!doctype html>
* <html>
* <head>
*     <title>Example Domain</title>
*
*     <meta charset="utf-8" />
*     <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
*     <meta name="viewport" content="width=device-width, initial-scale=1" />
*     <style type="text/css">
*     body {
*         background-color: #f0f0f2;
*         margin: 0;
*         padding: 0;
*         font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
*         
*     }
*     div {
*         width: 600px;
*         margin: 5em auto;
*         padding: 2em;
*         background-color: #fdfdff;
*         border-radius: 0.5em;
*         box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);
*     }
*     a:link, a:visited {
*         color: #38488f;
*         text-decoration: none;
*     }
*     @media (max-width: 700px) {
*         div {
*             margin: 0 auto;
*             width: auto;
*         }
*     }
*     </style>    
* </head>
*
* <body>
* <div>
*     <h1>Example Domain</h1>
*     <p>This domain is for use in illustrative examples in documents. You may use this
*     domain in literature without prior coordination or asking for permission.</p>
*     <p><a href="https://www.iana.org/domains/example">More information...</a></p>
* </div>
* </body>
* </html>
*
error: No query result
  --> sample.hurl:7:7
   |
 7 | name: variable "in" regex "Hello ([a-zA-Z]+)!"
   |       ^^^^^^^^^^^^^ The query didn't return any result
   |

But when you go to the actual webpage https://example.org/helloworld, it doesn't actually have "Hello name!" (where name is someones name) - so no match is expected.

However, if I modify the example hurl to be:

GET https://example.org/helloworld

HTTP/1.0 200
[Captures]
in: body
name: variable "in" regex "Example ([a-zA-Z]+)"

I am expecting it to return name: Domain - but it gives the same issue:

error: No query result
  --> sample.hurl:7:7
   |
 7 | name: variable "in" regex "Example ([a-zA-Z]+)"
   |       ^^^^^^^^^^^^^ The query didn't return any result
   |

In fact, if I do name: variable "in" regex "(.+)", I still don't get a match.

So the two things:

1. Can the example code be written to actually work with the content on https://example.org/helloworld, and
2. Am I doing something wrong with the variable stuff, and if so, can the docs reflect the right way too?

Thanks :)
Luke

[jcamiel]

Hi, thanks for the feedback.

1. for the moment, all examples are "fictive" and non meant to be running. I understand this is confusing and we may change change sometimes in the future (not soon though, there are a lot of samples in our doc!)
2. you actually found a bug with the query variable. It doesn't update the variables set on the current entry. For instance, if you run your example "in two passes", it is ok:

GET https://example.org/helloworld

HTTP/* *
[Captures]
in: body # the query variable "in" will be working in the next entry 


GET https://example.org/helloworld

HTTP/* *
[Captures]
name: variable "in" regex "Example ([a-zA-Z]+)"

(I've changed the status code for the sample to work)

Note that in this particular case you can capture the body directly:

GET https://example.org/helloworld

HTTP/* *
[Captures]
name: body name regex "Example ([a-zA-Z]+)")

I will fill two issues, one enhancement for the doc and one bug for your sample,

Thanks!