Experiment with path- and context-sensitivity for memLeak analysis

[michael-schwarz]

I'll use this issue to document some of the steps towards coming up with a good config for sv-comp.

Machine Config

parallel runs:           40
resource limits:
- memory:                3000.0 MB
- time:                  1800 s
- cpu cores:             1
hardware requirements:
- cpu cores:             1
- memory:                3000.0 MB
------------------------------------------------------------

   SYSTEM INFORMATION
host:                    goblint2
os:                      Linux-5.15.0-84-generic-x86_64-with-glibc2.29
cpu:                     Intel Xeon Platinum 8260 CPU @ 2.40GHz
- cores:                 96
- max frequency:         3900.0 MHz
- turbo boost enabled:   True
ram:                     540649.373696 MB
------------------------------------------------------------

Results without any additional path- & context-sensitivity:

Statistics:           3862 Files
  correct:            1082
    correct true:     1082
    correct false:       0
  incorrect:             0
    incorrect true:      0
    incorrect false:     0
  unknown:            2780
  Score:              2164 (max: 5857)

Results with full path- & context-sensitivity:

Statistics:           3862 Files
correct:            1104
  correct true:     1104
  correct false:       0
incorrect:             0
  incorrect true:      0
  incorrect false:     0
unknown:            2758
Score:              2208 (max: 5857)

Results with full path- & context-sensitivity & unique count of 5:

Statistics:           3862 Files
correct:            1131
  correct true:     1131
  correct false:       0
incorrect:             0
  incorrect true:      0
  incorrect false:     0
unknown:            2731
Score:              2262 (max: 5857)

Results with full path- & context-sensitivity & unique count of 5 & fix for #1235 :

Statistics:           3862 Files
  correct:            1158
    correct true:     1158
    correct false:       0
  incorrect:             0
    incorrect true:      0
    incorrect false:     0
  unknown:            2704
  Score:              2316 (max: 5857)

[michael-schwarz]

When being fully path- & context-sensitive, we succeed on 22 additional tasks.

results.2023-11-03_13-56-44.table.zip

[michael-schwarz]

We get strictly better with each of these config options, so it seems clear we would want them.

results.2023-11-06_10-15-02.zip

[michael-schwarz]

To facilitate development and get accurate pre-runs, I'd propose merging this now, so we don't have multiple PRs floating around.

[michael-schwarz]

Quick question @sim642: I saw you specify svcomp24.json in the artifact. Is this a symlink to svcomp.json for now, or do I need to make these changes also in some other places?

[sim642]

Quick question @sim642: I saw you specify svcomp24.json in the artifact. Is this a symlink to svcomp.json for now, or do I need to make these changes also in some other places?

It's a config I copied and adapted on the svcomp24-dev branch before some of the PRs were merged.

This PR doesn't even change svcomp.json, so it shouldn't matter. Or should the path-sensitivity not be default but only in SV-COMP conf?

[michael-schwarz]

Ok, thanks! Are you merging master back into svcomp24-dev regularly then?

Or should the path-sensitivity not be default but only in SV-COMP conf?

No, I think it makes sense to have it path-sensitive by default.