Skip to content

fix: support named captures #44

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kou merged 22 commits into from
Sep 18, 2022
Merged

fix: support named captures #44

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
16a4b46
feat: add test for named captures
ericgpks Jul 7, 2022
a3a7de3
fix: change test to be
ericgpks Jul 7, 2022
aecf835
feat: add empty method
ericgpks Jul 12, 2022
f9e9510
Update ext/strscan/strscan.c
ericgpks Jul 14, 2022
cd5c2c5
feat: change to use onig_foreach_name
ericgpks Aug 31, 2022
d8178c0
Merge branch 'master' into fix-support-named-captures
ericgpks Sep 13, 2022
1e1bf0f
fix: prepare another method to use main and change main logic
ericgpks Sep 13, 2022
666f22b
Merge branch 'fix-support-named-captures' of https://github.com/ericg…
ericgpks Sep 13, 2022
7380bda
feat: add document
ericgpks Sep 13, 2022
83d1655
Update ext/strscan/strscan.c
ericgpks Sep 14, 2022
617c3f9
Update test/strscan/test_stringscanner.rb
ericgpks Sep 14, 2022
83f042e
Update ext/strscan/strscan.c
ericgpks Sep 14, 2022
aecf844
Update test/strscan/test_stringscanner.rb
ericgpks Sep 14, 2022
30fb1b4
fix regex format in test
ericgpks Sep 14, 2022
f63639c
fix: change to use assert to check boolean
ericgpks Sep 14, 2022
aa60ea8
Update test/strscan/test_stringscanner.rb
ericgpks Sep 14, 2022
a139d4d
fix: change test to check length of a string
ericgpks Sep 16, 2022
4ed10e4
Merge branch 'fix-support-named-captures' of https://github.com/ericg…
ericgpks Sep 16, 2022
171a940
fix: change order to test expected and actual
ericgpks Sep 17, 2022
af71c09
fix: change expected value to use literal
ericgpks Sep 17, 2022
5b11f73
Fix style
kou Sep 18, 2022
5def9c2
Simplify
kou Sep 18, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions ext/strscan/strscan.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1458,6 +1458,26 @@ strscan_fixed_anchor_p(VALUE self)
return p->fixed_anchor_p ? Qtrue : Qfalse;
}

/*
* call-seq:
* scanner.named_captures -> hash
*
*
*/
static VALUE
strscan_named_captures(VALUE self)
{
struct strscanner *p;
VALUE pattern;
GET_SCANNER(self, p);
int i = 0;
int z = 0;
VALUE captures = rb_hash_new();
onig_foreach_name(RREGEXP(RMATCH(RREGEXP_PTR(pattern))->regexp)->ptr, i, z);

@kou kou Aug 31, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is a template for the next step:

typedef struct {
    struct strscanner *scanner;
    VALUE captures;
} named_captures_data;

static int
named_captures_iter(const OnigUChar *name,
                    const OnigUChar *name_end,
                    int back_num,
                    int *back_refs,
                    OnigRegex regex,
                    void *arg)
{
    named_captures_data *data = arg;
    return 0;
}

/*
 * call-seq:
 *   scanner.named_captures -> hash
 *
 *
 */
static VALUE
strscan_named_captures(VALUE self)
{
    struct strscanner *p;
    GET_SCANNER(self, p);
    named_captures_data data;
    data.scanner = p;
    data.captures = rb_hash_new();
    onig_foreach_name(RREGEXP_PTR(p->regex), named_captures_iter, &data);

    return data.captures;
}

I'll explain this later...

@kou kou Sep 1, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry. I explain this in Japanese...

onig_foreach_name()の第2引数は各名前ごとに実行される関数を指定します。たとえば、/(?<a>.)(?<b>.)/ならaに対して1回、bに対して1回、計2回指定した関数が実行されます。第3引数はその関数が実行されるときにその関数にそのまま渡されます。↑のコードで言えば、第3引数に指定した&dataがnamed_captures_iter()の最後の引数void *argになります。

では、なぜこんなことをしないといけないのでしょうか。それは、Cにはクロージャー機能がないからです。

Rubyにはクロージャー機能があるので次のようにブロックの中からブロックの外にある変数にアクセスできます。

a = 1
[1, 2, 3].each do |x|
  p [a, x]
end

しかし、Cにはクロージャー機能がないのでブロックの外の変数にアクセスできません。ここで言えばnamed_captures_iter()の中から結果を入れるために用意したrb_hash_new()（VALUE capturesに入っているやつ）にはアクセスできません。そのため、関数内で使いたい値は明示的に渡さないといけません。そういうことができるようにするためにonig_foreach_name()の第3引数があります。

さらにしかし！今回は渡したい値が2つあります。struct strscanner *pとVALUE captures = rb_hash_new()です。あ、いや、strcut strscanner *pじゃだめだな。。。VALUE selfの方じゃないとだめだな。。。

さーせん、やっぱりこんな感じで。

typedef struct {
    VALUE self;
    VALUE captures;
} named_captures_data;

static int
named_captures_iter(const OnigUChar *name,
                    const OnigUChar *name_end,
                    int back_num,
                    int *back_refs,
                    OnigRegex regex,
                    void *arg)
{
    named_captures_data *data = arg;
    return 0;
}

/*
 * call-seq:
 *   scanner.named_captures -> hash
 *
 *
 */
static VALUE
strscan_named_captures(VALUE self)
{
    struct strscanner *p;
    GET_SCANNER(self, p);
    named_captures_data data;
    data.self = self;
    data.captures = rb_hash_new();
    onig_foreach_name(RREGEXP_PTR(p->regex), named_captures_iter, &data);

    return data.captures;
}

で、複数の値を渡したいときは1つの値にしないといけないんです。そのためのCの機能がstructです。Rubyで言えばインスタンス変数だけがあるクラスみたいなもんです。structを使うと複数の値を1つにまとめることができます。なので、↓のようにすると1つの値にまとめることができます。

typedef struct {
    VALUE self;
    VALUE captures;
} named_captures_data;

ということで、structを使ってselfとcapturesをnamed_captures_iter()に渡しています。

onig_foreach_name(RREGEXP_PTR(p->regex), named_captures_iter, &data);の&dataの&ってなに！？と思うと思いますが、ポインターの話になるので、今回はなにも考えずに&を使っておいてください。

で、この渡した値をどうやってnamed_captures_iter()で受け取るかと言うと

    named_captures_data *data = arg;

です。void *argなのでargはvoid *型なのですが、void *型は単なるポインター型です。一方、named_captures_data *もポインター型なのですが、named_captures_dataのポインター型です。void *型は任意のポインター型に変換できるので

    named_captures_data *data = arg;

で、named_captures_dataのポインターとして扱うよという意味になります。argがonig_foreach_name(RREGEXP_PTR(p->regex), named_captures_iter, &data);の&dataに相当するので、onig_foreach_name()に渡した値を元に戻しているくらいに思ってもらえれば十分です。

ここまでは、とりあえず、大丈夫ですか？

ここまでわかったら後はnamed_captures_iter()の中身を実装していくだけになります。

具体的にはこんな感じになるんですが、後でまた説明するので、とりあえず↑まででわからないところがないか確認してもらえますか！？

static int
named_captures_iter(const OnigUChar *name,
                    const OnigUChar *name_end,
                    int back_num,
                    int *back_refs,
                    OnigRegex regex,
                    void *arg)
{
    named_captures_data *data = arg;

    VALUE key = rb_str_new((const char *)name, name_end - name);
    VALUE value = RUBY_Qnil;
    int i;
    for (i = 0; i < back_num; i++) {
        value = strscan_aref(data->self, INT2NUM(back_refs[i]));
    }
    rb_hash_aset(data->captures, key, value);

    return 0;
}


return captures;
}
Comment thread
ericgpks marked this conversation as resolved.

/* =======================================================================
Ruby Interface
======================================================================= */
Expand Down Expand Up @@ -1652,4 +1672,6 @@ Init_strscan(void)
rb_define_method(StringScanner, "inspect", strscan_inspect, 0);

rb_define_method(StringScanner, "fixed_anchor?", strscan_fixed_anchor_p, 0);

rb_define_method(StringScanner, "named_captures", strscan_named_captures, 0);
}
8 changes: 8 additions & 0 deletions test/strscan/test_stringscanner.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -763,6 +763,14 @@ def test_fixed_anchor_false
assert_equal(false, StringScanner.new("a", fixed_anchor: nil).fixed_anchor?)
assert_equal(false, StringScanner.new("a", fixed_anchor: false).fixed_anchor?)
end

def test_named_captures
Comment thread
ericgpks marked this conversation as resolved.
s = "foobarbaz"
re = /(?<f>foo)(?<r>bar)(?<z>baz)/
scan = StringScanner.new(s)
scan.match? re

@kou kou Sep 13, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can use assert {...} for boolean expression to get better failure message:

Suggested change
scan.match? re
assert do
scan.match?(re)
end

@ericgpks ericgpks Sep 14, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it to be assert scan.match?(re) ?

@kou kou Sep 15, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, no. assert scan.match?(re) and assert {scan.match?(re)} is difference. And the latter is preferred for better message on failure. See also: https://github.com/ruby/power_assert

But we can't use assert {...} here. Sorry.
We can use assert {...} in ruby/strscan but we can't use assert {...} in ruby/ruby. Because ruby/ruby doesn't use test-unit gem.

So we should use assert_true(scan.match?(re)) here.

@ericgpks ericgpks Sep 16, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that scan.match?(re) returns a length of matching. So can I change this test to check length of string?

@kou kou Sep 16, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow! I thought that it returns a boolean value because its name ends with ?.

You're right. We should use assert_equal(LENGTH, scan.match?(re)).

@ericgpks ericgpks Sep 17, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed this test.

Comment thread
ericgpks marked this conversation as resolved.
Outdated
assert_equal({"f"=>"foo", "r"=>"bar", "z"=>"baz"}, scan.named_captures)
Comment thread
ericgpks marked this conversation as resolved.
Outdated
end
end

class TestStringScannerFixedAnchor < TestStringScanner
Expand Down