Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

capture returns inconsistent results for optional named group #3093

Open
elmaimbo opened this issue Apr 15, 2024 · 2 comments
Open

capture returns inconsistent results for optional named group #3093

elmaimbo opened this issue Apr 15, 2024 · 2 comments

Comments

@elmaimbo
Copy link

Describe the bug
I've just upgraded from JQ version 1.6 to 1.7, and noticed that the capture function is returning an empty string instead of null for an optional named group that doesn't match, but only when no other part of the regex matches.

For example, if your regex contains (?<x>a)?, then if the overall regular expression matches, the output from capture will contain a field x which should either have the value "a" if the capturing group is present, or null if it isn't. The problem is that there are some cases where x has the value "" (i.e. empty string).

To Reproduce
Running the following code shows the issue:
jq -cn '"a","b","c" | capture("(?<x>a)?b?")'

The third line that is output from the command above is wrong because there is no valid case where x can have the value "".

Expected behavior
A capturing group that is followed by "?" should have the value null if the capturing group isn't present.

i.e. The expected result from running the example code above should be:

{"x":"a"}
{"x":null}
{"x":null}

However the output produced by JQ versions 1.7 and 1.7.1 are:

{"x":"a"}
{"x":null}
{"x":""}

i.e. The third line produced an x field with the value "" instead of null.

(FYI JQ 1.6 produces {} as the third line of output, which is also arguably wrong, but IMHO is better than what JQ 1.7 produces.)

Environment (please complete the following information):

  • OS and Version: Linux Ubuntu 23.10
  • jq version 1.7
@wader
Copy link
Member

wader commented Apr 15, 2024

The change from empty object to object with empty string seem to be have happen in 01dfd8b

The comment in the code says it's about "Zero-width match" but that dosen't seem to be true in this case? the group did not match at all or?

I wonder if the code here https://github.com/jqlang/jq/blob/master/src/builtin.c#L931 should do the same region->beg[i] == -1 check as the code below? is that how oniguruma signals that a group did not match?

@wader
Copy link
Member

wader commented Apr 15, 2024

With:

diff --git a/src/builtin.c b/src/builtin.c
index ebc1863..0fbdfdf 100644
--- a/src/builtin.c
+++ b/src/builtin.c
@@ -930,8 +930,13 @@ static jv f_match(jq_state *jq, jv input, jv regex, jv modifiers, jv testmode) {
         jv captures = jv_array();
         for (int i = 1; i < region->num_regs; ++i) {
           jv cap = jv_object();
-          cap = jv_object_set(cap, jv_string("offset"), jv_number(idx));
-          cap = jv_object_set(cap, jv_string("string"), jv_string(""));
+          if (region->beg[i] == -1) {
+            cap = jv_object_set(jv_object(), jv_string("offset"), jv_number(-1));
+            cap = jv_object_set(cap, jv_string("string"), jv_null());
+          } else {
+            cap = jv_object_set(cap, jv_string("offset"), jv_number(idx));
+            cap = jv_object_set(cap, jv_string("string"), jv_string(""));
+          }
           cap = jv_object_set(cap, jv_string("length"), jv_number(0));
           cap = jv_object_set(cap, jv_string("name"), jv_null());
           captures = jv_array_append(captures, cap);

I get:

$ ./jq -cn '"a","b","c" | capture("(?<x>a)?b?")'
{"x":"a"}
{"x":null}
{"x":null}

$ ./jq -cn '"a","b","c" | match("(?<x>a)?b?")'
{"offset":0,"length":1,"string":"a","captures":[{"offset":0,"length":1,"string":"a","name":"x"}]}
{"offset":0,"length":1,"string":"b","captures":[{"offset":-1,"string":null,"length":0,"name":"x"}]}
{"offset":0,"length":0,"string":"","captures":[{"offset":-1,"string":null,"length":0,"name":"x"}]}

# with group that allows empty match

$ ./jq -cn '"a","b","c" | capture("(?<x>a?)?b?")'
{"x":"a"}
{"x":""}
{"x":""}

$ ./jq -cn '"a","b","c" | match("(?<x>a?)?b?")'
{"offset":0,"length":1,"string":"a","captures":[{"offset":0,"length":1,"string":"a","name":"x"}]}
{"offset":0,"length":1,"string":"b","captures":[{"offset":0,"string":"","length":0,"name":"x"}]}
{"offset":0,"length":0,"string":"","captures":[{"offset":0,"string":"","length":0,"name":"x"}]}

But this changes how "abc" | [match("( )*"; "g")] behaves in this test https://github.com/jqlang/jq/blob/master/tests/onig.test#L1-L4 new result is [{"offset":0,"length":0,"string":"","captures":[{"offset":-1,"string":null,"length":0,"name":null}]},{"offset":1,"length":0,"string":"","captures":[{"offset":-1,"string":null,"length":0,"name":null}]},{"offset":2,"length":0,"string":"","captures":[{"offset":-1,"string":null,"length":0,"name":null}]},{"offset":3,"length":0,"string":"","captures":[{"offset":-1,"string":null,"length":0,"name":null}]}]
but i'm not sure i follow how the current expected result is correct? why would the capture group that didn't match have an offset?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants