[interpreter/ruby]: Optimize ep check to simplify work for verifier, increase limit

[dalehamel]

What

Optimizes the inner loop used when walking from the cfp->ep until we hit a local env, when checking for a method entry.

I also realized we had a regression on MAX_EP_CHECKS, on our branch I had raised it to 10 but in the open-telemetry#907 it was only set to 6. I addressed that issue, as we have some cases (ruby graphql libraries in rails) that seem to potentially have up to 10 levels of indirection to get to the IMEMO_MENT entry. It will now walk up to 10 environment pointers before giving up (and erroring), instead of just 6.

Why

Even though it passed the verifier in CI, when I rebased after merging and tried in production, I started having verifier errors. It works on all CI amd64 checks, works locally on aarch64 with linux 6.8, but fails with amd64 on linux 6.6.

It seems that when I switched from goto's to a simpler UNROLL while addressing the feedback in open-telemetry#907 , I didn't optimally structure the loop.

Without these changes, it will error¹ out if there is a lot of indirection to find a local method entry, such as in ruby graphql libraries and rails with a lot of meta-programming.

How

I tried to push all code that didn't need to be in the unroll to either before or after it. The unroll will now only check:

• If it is local (base case break the recursion / stop the loop)
• read the imemo from rbasic flags
• and always exit if it is a method entry
• break the loop if it is local and an svar

All other probe reads and cfunc checks are moved out of the loop explicitly.

The MAX_EP_CHECKS macro is bumped to 10

Footnotes

1. perhaps bailing on unwinding the whole stack is too severe, we can probably unwind the other frames and if we hit max ep checks, return a dummy frame for userpace to symbolize as something like <unknown ruby_local_ep_missing> ↩

[dalehamel]

Upstream PR opened open-telemetry#1100

[dalehamel]

merged upstream