Skip to content

Commit 21d4a76

Browse files
authored
Merge pull request #38 from ispras/script_for_traces_filtering
Add a script for filtering error traces
2 parents e014053 + 3fa78ac commit 21d4a76

File tree

8 files changed

+171
-22
lines changed

8 files changed

+171
-22
lines changed

.github/workflows/deploy.yml

+23-3
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
name: Deploy
22

3-
on: [push, pull_request, workflow_dispatch]
3+
on: [push, workflow_dispatch]
44

55
jobs:
66
build-launcher:
77
runs-on: ubuntu-latest
88
strategy:
99
matrix:
10-
python-version: ["3.8", "3.9", "3.10"]
10+
python-version: ["3.8"]
1111
steps:
1212
- uses: actions/checkout@v3
1313
- name: Set up Python ${{ matrix.python-version }}
@@ -31,7 +31,7 @@ jobs:
3131
runs-on: ubuntu-latest
3232
strategy:
3333
matrix:
34-
python-version: ["3.8", "3.9", "3.10"]
34+
python-version: ["3.8"]
3535
steps:
3636
- uses: actions/checkout@v3
3737
- name: Set up Python ${{ matrix.python-version }}
@@ -48,6 +48,26 @@ jobs:
4848
cd build
4949
python3 ./scripts/visualize_witnesses.py -r results/ -d ../docs/examples/witnesses/violation/
5050
python3 ./scripts/visualize_witnesses.py -r results/ -d ../docs/examples/witnesses/correctness/
51+
build-mea:
52+
runs-on: ubuntu-latest
53+
strategy:
54+
matrix:
55+
python-version: ["3.8"]
56+
steps:
57+
- uses: actions/checkout@v3
58+
- name: Set up Python ${{ matrix.python-version }}
59+
uses: actions/setup-python@v3
60+
with:
61+
python-version: ${{ matrix.python-version }}
62+
- name: Install dependencies
63+
run: |
64+
python3 -m pip install --upgrade pip
65+
pip3 install requests ujson graphviz ply pytest atomicwrites more-itertools pluggy py attrs setuptools six django==2.1 psycopg2 pycparser sympy
66+
- name: Deployment of MEA
67+
run: |
68+
DEPLOY_DIR=build make install-mea -j$(nproc)
69+
cd build
70+
python3 ./scripts/mea.py -d ../docs/examples/witnesses/violation/
5171
build-frama-c-cil:
5272
runs-on: ubuntu-latest
5373
strategy:

.github/workflows/pylint.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
name: Pylint
22

3-
on: [push, pull_request, workflow_dispatch]
3+
on: [push, workflow_dispatch]
44

55
jobs:
66
build:
77
runs-on: ubuntu-latest
88
strategy:
99
matrix:
10-
python-version: ["3.8", "3.9", "3.10"]
10+
python-version: ["3.8"]
1111
steps:
1212
- uses: actions/checkout@v3
1313
- name: Set up Python ${{ matrix.python-version }}

Makefile

+6
Original file line numberDiff line numberDiff line change
@@ -231,6 +231,12 @@ install-witness-visualizer: check-deploy-dir build-klever
231231
cp ${klever_dir}/bridge/reports/mea/core.py scripts/aux/mea.py
232232
@echo "*** Witness Visualizer has been successfully installed into the directory ${DEPLOY_DIR} ***"
233233

234+
install-mea: install-witness-visualizer
235+
@cd ${DEPLOY_DIR} ; \
236+
rm -f scripts/${wv_script} ; \
237+
rm -rf ${klever_dir}/bridge/
238+
@echo "*** MEA has been successfully installed into the directory ${DEPLOY_DIR} ***"
239+
234240
install-benchmark-visualizer: install-witness-visualizer
235241
@cp -r ${klever_dir}/utils/ ${DEPLOY_DIR}/${klever_dir}/
236242
@cp -f ${root_dir}/scripts/${bv_script} ${DEPLOY_DIR}/scripts/

README.md

+22-2
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ make install-witness-visualizer DEPLOY_DIR=<deployment directory>
3030
After deployment Witness Visualizer can be used to convert witnesses from the `<deployment directory>` with command:
3131

3232
```
33-
scripts/visualize_witnesses.py OPTIONS
33+
<deployment directory>/scripts/visualize_witnesses.py OPTIONS
3434
```
3535

3636
Primary options:
@@ -44,7 +44,7 @@ Primary options:
4444
For example:
4545

4646
```bash
47-
scripts/visualize_witnesses.py --witness output/witness.graphml --result-dir results/ --source-dir ~/sv-benchmarks
47+
<deployment directory>/scripts/visualize_witnesses.py --witness output/witness.graphml --result-dir results/ --source-dir ~/sv-benchmarks
4848
```
4949

5050
There are some examples of [SV-COMP](https://sv-comp.sosy-lab.org) witnesses in the `docs/examples/witnesses` directory,
@@ -71,3 +71,23 @@ make install-benchmark-visualizer DEPLOY_DIR=<deployment directory>
7171
#### Usage
7272

7373
See instruction [docs/benchmark_visualizer.md](docs/benchmark_visualizer.md).
74+
75+
## Filtering of witnesses
76+
77+
Multiple Error Analysis (MEA) stands for semi-automatic violation witnesses filtering.
78+
This framework provides part of MEA, which performs automatic filtering.
79+
In order to do it, some core elements are extracted from a violation witness by means of
80+
a specified `conversion` function and then compared with core elements of other witnesses
81+
by means of a specified `comparison` function.
82+
83+
#### Deployment
84+
MEA library can be installed in the `<deployment directory>` with the following command:
85+
```shell
86+
make install-mea DEPLOY_DIR=<deployment directory>
87+
```
88+
89+
#### Usage
90+
```shell
91+
<deployment directory>/scripts/mea.py -d <directory with violation witnesses>
92+
```
93+
All unique violation witnesses will be printed as a result.

scripts/auto_check.py

+7-4
Original file line numberDiff line numberDiff line change
@@ -86,11 +86,14 @@ def __init__(self, config_file: str):
8686
self.output_desc = sys.stdout
8787
else:
8888
self.output_desc = subprocess.DEVNULL
89-
logger_level = logging.DEBUG if self.debug else logging.INFO
90-
logging.basicConfig(format='%(asctime)s: %(name)s: %(levelname)s: %(message)s',
91-
level=logger_level, datefmt='%Y-%m-%d %H:%M:%S')
89+
9290
self.logger = logging.getLogger(name=COMPONENT_AUTO_CHECKER)
93-
self.logger.setLevel(logger_level)
91+
stream_handler = logging.StreamHandler(stream=sys.stdout)
92+
stream_handler.setFormatter(logging.Formatter(
93+
'%(asctime)s: %(name)s: %(levelname)s: %(message)s', '%Y-%m-%d %H:%M:%S')
94+
)
95+
self.logger.addHandler(stream_handler)
96+
self.logger.setLevel(logging.DEBUG if self.debug else logging.INFO)
9497

9598
self.hostname = None
9699
self.hostname = self.__command_caller("hostname", get_stdout=True)

scripts/components/component.py

+14-5
Original file line numberDiff line numberDiff line change
@@ -69,17 +69,17 @@ def __init__(self, name: str, config: dict):
6969

7070
# Debug and logging.
7171
self.debug = self.component_config.get(TAG_DEBUG, False)
72-
logger_level = logging.DEBUG if self.debug else logging.INFO
73-
logging.basicConfig(format='%(name)s: %(levelname)s: %(message)s', level=logger_level)
74-
self.logger = logging.getLogger(name=self.name)
75-
self.logger.setLevel(logger_level)
72+
self.logger = self._create_logger(self.name, logging.DEBUG if self.debug else logging.INFO)
7673

7774
# Should be rewritten.
7875
self.install_dir = None
7976
self.error_logs = set()
8077
self.temp_logs = set()
8178
if not Component.tools_config:
82-
with open(TOOL_CONFIG_FILE, encoding='ascii') as file_obj:
79+
install_dir = os.path.abspath(
80+
os.path.join(os.path.dirname(os.path.realpath(__file__)), os.pardir, os.pardir)
81+
)
82+
with open(os.path.join(install_dir, TOOL_CONFIG_FILE), encoding='ascii') as file_obj:
8383
Component.tools_config = json.load(file_obj)
8484

8585
def __propagate_config(self):
@@ -93,6 +93,15 @@ def __propagate_config(self):
9393
component_config[tag] = self.config[tag]
9494
self.component_config = component_config
9595

96+
@staticmethod
97+
def _create_logger(logger_name: str, logger_level):
98+
logger = logging.getLogger(name=logger_name)
99+
stream_handler = logging.StreamHandler(stream=sys.stdout)
100+
stream_handler.setFormatter(logging.Formatter('%(name)s: %(levelname)s: %(message)s'))
101+
logger.addHandler(stream_handler)
102+
logger.setLevel(logger_level)
103+
return logger
104+
96105
def runexec_wrapper(self, cmd, output_dir=None, output_file=None):
97106
"""
98107
Call a command inside RunExec tool, track its resources and redirect its output into

scripts/components/mea.py

+3-6
Original file line numberDiff line numberDiff line change
@@ -352,12 +352,9 @@ def __parse_trace(self, error_trace_file: str, supported_types: set) -> dict:
352352
from core.vrp.et import import_error_trace
353353

354354
# Those messages are waste of space.
355-
logger = logging.getLogger(name="Witness processor")
356-
logging.basicConfig(format='%(name)s: %(levelname)s: %(message)s')
357-
if self.debug:
358-
logger.setLevel(logging.WARNING)
359-
else:
360-
logger.setLevel(logging.ERROR)
355+
logger = self._create_logger(
356+
"Witness processor", logging.WARNING if self.debug else logging.ERROR
357+
)
361358
try:
362359
json_error_trace = import_error_trace(logger, error_trace_file, self.source_dir)
363360
if self.dry_run:

scripts/mea.py

+94
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
#!/usr/bin/python3
2+
#
3+
# CV is a framework for continuous verification.
4+
#
5+
# Copyright (c) 2018-2023 ISP RAS (http://www.ispras.ru)
6+
# Ivannikov Institute for System Programming of the Russian Academy of Sciences
7+
#
8+
# Licensed under the Apache License, Version 2.0 (the "License");
9+
# you may not use this file except in compliance with the License.
10+
# You may obtain a copy of the License at
11+
#
12+
# http://www.apache.org/licenses/LICENSE-2.0
13+
#
14+
# Unless required by applicable law or agreed to in writing, software
15+
# distributed under the License is distributed on an "AS IS" BASIS,
16+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
17+
# See the License for the specific language governing permissions and
18+
# limitations under the License.
19+
#
20+
21+
"""
22+
This script provides functionality to filter a given witnesses.
23+
"""
24+
25+
import argparse
26+
27+
from components.mea import *
28+
29+
30+
def _create_config(options=None, conversion_function="", comparison_function=""):
31+
additional_mf = []
32+
is_debug = False
33+
if options:
34+
conversion_function = options.conversion
35+
comparison_function = options.comparison
36+
additional_mf = options.mf or []
37+
is_debug = options.debug
38+
return {
39+
COMPONENT_MEA: {
40+
TAG_COMPARISON_FUNCTION: comparison_function,
41+
TAG_CONVERSION_FUNCTION: conversion_function,
42+
TAG_CONVERSION_FUNCTION_ARGUMENTS: {
43+
TAG_ADDITIONAL_MODEL_FUNCTIONS: additional_mf
44+
},
45+
TAG_DEBUG: is_debug,
46+
TAG_CLEAN: False,
47+
TAG_UNZIP: False,
48+
TAG_DRY_RUN: False,
49+
TAG_SOURCE_DIR: None
50+
}
51+
}
52+
53+
54+
def _parse_cmdline() -> tuple:
55+
parser = argparse.ArgumentParser()
56+
parser.add_argument("-d", "--directory", help="directory with witnesses to be filtered",
57+
required=True)
58+
parser.add_argument("--conversion", help="conversion function",
59+
default=DEFAULT_CONVERSION_FUNCTION)
60+
parser.add_argument("--comparison", help="comparison function",
61+
default=DEFAULT_COMPARISON_FUNCTION)
62+
parser.add_argument("--additional-model-functions", dest='mf', nargs='+',
63+
help="additional model functions, separated by whitespace")
64+
parser.add_argument('--debug', action='store_true')
65+
options = parser.parse_args()
66+
67+
witnesses = glob.glob(os.path.join(options.directory, f"witness.*{GRAPHML_EXTENSION}"))
68+
return witnesses, _create_config(options)
69+
70+
71+
def execute_filtering(witnesses: list, config=None, conversion_function="",
72+
comparison_function="") -> list:
73+
"""
74+
Filter the given violation witnesses.
75+
"""
76+
if not config:
77+
config = _create_config(conversion_function=conversion_function,
78+
comparison_function=comparison_function)
79+
script_dir = os.path.join(os.path.dirname(os.path.realpath(__file__)), os.pardir)
80+
install_dir = os.path.abspath(os.path.join(script_dir, DEFAULT_INSTALL_DIR))
81+
82+
if not os.path.exists(install_dir):
83+
install_dir = os.path.abspath(os.path.join(os.pardir, DEFAULT_INSTALL_DIR))
84+
mea = MEA(config, witnesses, install_dir)
85+
mea.logger.debug(f"Received {len(witnesses)} witnesses")
86+
processed_witnesses = mea.filter()
87+
mea.logger.debug(f"Number of unique witnesses is {len(processed_witnesses)}")
88+
return processed_witnesses
89+
90+
91+
if __name__ == "__main__":
92+
m_witnesses, m_config = _parse_cmdline()
93+
for witness in execute_filtering(m_witnesses, m_config):
94+
print(f"Unique witness '{witness}'")

0 commit comments

Comments
 (0)