Record accessor helper

[repeatedly]

Implement record_accessor helper for nested record support.
Support two jsonpath like syntaxes.

• dot notation: $.key1.key2[0]
• bracket notation: $['key1'][0]['this.is.key2']
• single key: key

In this patch, I also update filter_grep to use this helper.

NOTE: I implement Accessor with normal way. If need the performance, I will change its implementaion to eval way: https://gist.github.com/repeatedly/ab553ed260cd080bd01ec71da9427312

[repeatedly]

@mururu Could you check this patch?

[repeatedly]

Bench result:

% ruby dig_bench.rb
       user     system      total        real
baseline     0.570000   0.000000   0.570000 (  0.573822)
expand args  1.100000   0.000000   1.100000 (  1.093701)
pre-eval     0.860000   0.000000   0.860000 (  0.867493)

[sonots]

I thought jsonpath supports $.key1[0]['this.is.key2'], but this patch does not?

[repeatedly]

@sonots yes. In fluentd case, mix case seems no merits.

[sonots]

Well, I feel allowing users to write $.key1[0]['this.is.key2'] is a merit. As a user, I want to use bracket notation for only part having . (this.is.key2).

I agree that it makes parsing a bit complex and consumes time, but I think it is acceptable tradeoff because parser usually runs only once at "configure".

[repeatedly]

I think it is acceptable tradeoff because parser usually runs only once at "configure".

Yes, it doesn't have a impact at runtime.

This is just jsonpath like, not jsonpath, so I don't think following similar syntax is not must. For documentation and maintainance, separating two way is less cost at the baseline.

[sonots]

We need to get users' opinions. As one user, I like to get allowed $.key1[0]['this.is.key2'].

[sonots]

Wow, this is acceptable? It looks wierd for me. I think we should not allow it.

[repeatedly]

Why?

[sonots]

I think typical jsonpath parser does not accept this. At least https://github.com/json-path/JsonPath did not accept this.

[repeatedly]

It seems javascript object limitation. javascript object field can't contain space.
But fluentd target is an event record, so there is no such limitation.

[sonots]

I am against to extend specifications because it makes difficult to refactor implementation later (for example, think of changing jsonpath parser to 3rd party implementation later) without breaking backward compatibility.

[repeatedly]

We don't have a plan to use 3rd party jsonpath parser because this is just jsonpath like, not jsonpath. jsonpath has lots of useless feature in fluentd, e.g.g filter, wildcard and etc.
So we want limited well-known syntax, not entire jsonpath.

[repeatedly]

But limiting several characters in dot notation is not bad idea because bracket notation is straight way for such cases.

[mururu]

I don't strongly oppose this, but it just looks weird for me too.

[mururu]

$.hello.[0] is parsed as ["hello", "[0]", 0]. It should be parsed as ["hello", 0] or ["hello", "[0]"] or just raise error.
https://github.com/joshbuddy/jsonpath parses it as ["hello", 0]. I haven't check other implementations.

[mururu]

Minor comment: When parsing $[] or $.hello[], it says invalid value for Integer(): "]", but it is better to say invalid value for Integer(): "" .

[repeatedly]

@mururu Good catch. I will add more validation.

[repeatedly]

Applied reviews and improve the performance for single key.
PTAL

[mururu]

LGTM