Make a PreparedScriptBenchmark class closer to the original

[Taritsyn]

Hello!

Main difference between the PreparedScriptBenchmark class and the original one is that the script is pre-compiled in the Setup method and is therefore not taken into account when measuring performance.

Initial version of the PreparedScriptBenchmark class shows very good result:

| Method                           | Mean      | Error    | StdDev   | Median    | Ratio | RatioSD | Gen0      | Gen1      | Allocated | Alloc Ratio |
|--------------------------------- |----------:|---------:|---------:|----------:|------:|--------:|----------:|----------:|----------:|------------:|
| ExecuteStringOnMultipleEngines   | 195.27 ms | 3.767 ms | 4.898 ms | 194.13 ms |  1.00 |    0.03 | 5000.0000 | 2000.0000 |   34.1 MB |        1.00 |
| ExecutePreparedOnMultipleEngines |  98.61 ms | 1.966 ms | 5.610 ms |  96.88 ms |  0.51 |    0.03 | 3000.0000 | 1000.0000 |  18.94 MB |        0.56 |

Version with my edits shows a result close to the original:

| Method                           | Mean     | Error   | StdDev   | Ratio | RatioSD | Gen0      | Gen1      | Allocated | Alloc Ratio |
|--------------------------------- |---------:|--------:|---------:|------:|--------:|----------:|----------:|----------:|------------:|
| ExecuteStringOnMultipleEngines   | 197.0 ms | 3.94 ms |  4.21 ms |  1.00 |    0.03 | 5000.0000 | 2000.0000 |   34.1 MB |        1.00 |
| ExecutePreparedOnMultipleEngines | 254.5 ms | 5.08 ms | 10.37 ms |  1.29 |    0.06 | 3000.0000 | 1000.0000 |  23.58 MB |        0.69 |

Perhaps this is due to the fact that every time the Engine.PrepareScript method is called, then a new instance of the Parser class is created.

[lahma]

But this is how the script preparation is intended to be used, you cache them in process as the preparation is allowed to be costly and take time, then you have optimized script that can be fed to the Engine multiple times. Ad-hoc running of scripts doesn't need preparation.

Am I missing something here?

[Taritsyn]

Hello, Marko!

When I wrote JsExecutionHeavyBenchmark, I assumed that the costs of the script pre-compilation would not exceed 30% and would be compensated by the rapid initialization of four instances of JS engines.

It seems that the long execution time of the Engine.PrepareScript method is caused by regular expression compilation (CompileRegex = true → RegExpParseMode.AdaptToCompiled). I rewrote this benchmark again and got the following results:

| Method                                                  | Mean     | Error   | StdDev  | Ratio | RatioSD | Gen0      | Gen1      | Allocated | Alloc Ratio |
|-------------------------------------------------------- |---------:|--------:|--------:|------:|--------:|----------:|----------:|----------:|------------:|
| ExecuteStringOnMultipleEngines                          | 201.7 ms | 3.96 ms | 6.17 ms |  1.00 |    0.04 | 5000.0000 | 2000.0000 |   34.1 MB |        1.00 |
| ExecutePreparedWithoutRegexCompilationOnMultipleEngines | 131.8 ms | 2.62 ms | 3.84 ms |  0.65 |    0.03 | 3000.0000 | 1000.0000 |  22.92 MB |        0.67 |
| ExecutePreparedWithRegexCompilationOnMultipleEngines    | 256.6 ms | 5.11 ms | 8.40 ms |  1.27 |    0.06 | 3000.0000 | 1000.0000 |  23.58 MB |        0.69 |

I consider the result in second row to be very good. I'm planning to disable regular expression compilation in the JavaScriptEngineSwitcher.Jint module.

[lahma]

@Taritsyn great findings! indeed, regex compile is time-consuming and not lightweight. Is there still something that you would like to see in Jint repo via this PR, just merge this as an example of scenarios?

[Taritsyn]

This PR is complete and can be merged.

[lahma]

Thank you!