add support for assert greater equal. Update metrics property

[shahzebsiddiqui]

Here is the stream benchmark take note of the output lines

stream_test/7305fc15: testing metric: copy if 7981.3 >= 5000.0
stream_test/7305fc15: testing metric: scale if 9484.8 >= 5000.0

(buildtest)  ~/Documents/github/buildtest/ [perf_status] buildtest bd -b tutorials/stream.yml
╭─────────────────────────────────────────── buildtest summary ───────────────────────────────────────────╮                                                                                                                                
│                                                                                                         │                                                                                                                                
│ User:               siddiq90                                                                            │                                                                                                                                
│ Hostname:           DOE-7086392.vpn-dhcp.lbl.gov                                                        │                                                                                                                                
│ Platform:           Darwin                                                                              │                                                                                                                                
│ Current Time:       2022/12/16 12:41:03                                                                 │                                                                                                                                
│ buildtest path:     /Users/siddiq90/Documents/github/buildtest/bin/buildtest                            │                                                                                                                                
│ buildtest version:  1.0                                                                                 │                                                                                                                                
│ python path:        /Users/siddiq90/.local/share/virtualenvs/buildtest-Ir4AdrfC/bin/python3             │                                                                                                                                
│ python version:     3.7.3                                                                               │                                                                                                                                
│ Configuration File: /Users/siddiq90/Documents/github/buildtest/buildtest/settings/config.yml            │                                                                                                                                
│ Test Directory:     /Users/siddiq90/Documents/github/buildtest/var/tests                                │                                                                                                                                
│ Report File:        /Users/siddiq90/Documents/github/buildtest/var/report.json                          │                                                                                                                                
│ Command:            /Users/siddiq90/Documents/github/buildtest/bin/buildtest bd -b tutorials/stream.yml │                                                                                                                                
│                                                                                                         │                                                                                                                                
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                                                
─────────────────────────────────────────────────────────────────────────────────────────────────────────  Discovering Buildspecs ─────────────────────────────────────────────────────────────────────────────────────────────────────────
                       Discovered buildspecs                       
╔═════════════════════════════════════════════════════════════════╗
║ buildspec                                                       ║
╟─────────────────────────────────────────────────────────────────╢
║ /Users/siddiq90/Documents/github/buildtest/tutorials/stream.yml ║
╚═════════════════════════════════════════════════════════════════╝


Total Discovered Buildspecs:  1
Total Excluded Buildspecs:  0
Detected Buildspecs after exclusion:  1
─────────────────────────────────────────────────────────────────────────────────────────────────────────── Parsing Buildspecs ────────────────────────────────────────────────────────────────────────────────────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/Users/siddiq90/Documents/github/buildtest/tutorials/stream.yml: VALID
Total builder objects created: 1
                                                                                               Builders by type=script                                                                                               
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ builder              ┃ type   ┃ executor           ┃ compiler ┃ nodes ┃ procs ┃ description                                                     ┃ buildspecs                                                      ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ stream_test/7305fc15 │ script │ generic.local.bash │ None     │ None  │ None  │ Run stream test with metrics example using assert greater equal │ /Users/siddiq90/Documents/github/buildtest/tutorials/stream.yml │
└──────────────────────┴────────┴────────────────────┴──────────┴───────┴───────┴─────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────────────── Building Test ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
stream_test/7305fc15: Creating test directory: /Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/stream/stream_test/7305fc15
stream_test/7305fc15: Creating the stage directory: /Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/stream/stream_test/7305fc15/stage
stream_test/7305fc15: Writing build script: /Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/stream/stream_test/7305fc15/stream_test_build.sh
────────────────────────────────────────────────────────────────────────────────────────────────────────────── Running Tests ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
Spawning 1 processes for processing builders
─────────────────────────────────────────────────────────────────────────────────────────────────────────────── Iteration 1 ───────────────────────────────────────────────────────────────────────────────────────────────────────────────
stream_test/7305fc15 does not have any dependencies adding test to queue
In this iteration we are going to run the following tests: [stream_test/7305fc15]
stream_test/7305fc15: Running Test via command: bash --norc --noprofile -eo pipefail stream_test_build.sh
stream_test/7305fc15: Test completed in 2.67788 seconds
stream_test/7305fc15: Test completed with returncode: 0
stream_test/7305fc15: Writing output file -  /Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/stream/stream_test/7305fc15/stream_test.out
stream_test/7305fc15: Writing error file - /Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/stream/stream_test/7305fc15/stream_test.err
stream_test/7305fc15: testing metric: copy if 7981.3 >= 5000.0
stream_test/7305fc15: testing metric: scale if 9484.8 >= 5000.0
                                                   Test Summary                                                    
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder              ┃ executor           ┃ status ┃ checks (ReturnCode, Regex, Runtime) ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ stream_test/7305fc15 │ generic.local.bash │ PASS   │ False False False                   │ 0          │ 2.67788 │
└──────────────────────┴────────────────────┴────────┴─────────────────────────────────────┴────────────┴─────────┘



Passed Tests: 1/1 Percentage: 100.000%
Failed Tests: 0/1 Percentage: 0.000%


Adding 1 test results to /Users/siddiq90/Documents/github/buildtest/var/report.json
Writing Logfile to: /Users/siddiq90/Documents/github/buildtest/var/logs/buildtest_jfnd4nfo.log

The metrics are captured based on regular expession and they are converted to int, float, or str. In this example we are capturing the 1st item from the list

(buildtest)  ~/Documents/github/buildtest/ [perf_status] buildtest inspect query -o stream_test 
──────────────────────────────────────────────────────────────────────────────────────────── stream_test/7305fc15-dceb-4c4e-b094-fadf4ed6a7b9 ─────────────────────────────────────────────────────────────────────────────────────────────
Executor: generic.local.bash
Description: Run stream test with metrics example using assert greater equal
State: PASS
Returncode: 0
Runtime: 2.67788 sec
Starttime: 2022/12/16 12:41:03
Endtime: 2022/12/16 12:41:06
Command: bash --norc --noprofile -eo pipefail stream_test_build.sh
Test Script: /Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/stream/stream_test/7305fc15/stream_test.sh
Build Script: /Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/stream/stream_test/7305fc15/stream_test_build.sh
Output File: /Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/stream/stream_test/7305fc15/stream_test.out
Error File: /Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/stream/stream_test/7305fc15/stream_test.err
Log File: /Users/siddiq90/Documents/github/buildtest/var/logs/buildtest_jfnd4nfo.log
     Metrics      
┏━━━━━━━┳━━━━━━━━┓
┃ Name  ┃ Value  ┃
┡━━━━━━━╇━━━━━━━━┩
│ copy  │ 7981.3 │
│ scale │ 9484.8 │
└───────┴────────┘
──────────────────────────────────────────────────── Output File: /Users/siddiq90/Documents/github/buildtest/var/tests/generic.local.bash/stream/stream_test/7305fc15/stream_test.out ─────────────────────────────────────────────────────
-------------------------------------------------------------                                                                                                                                                                              
STREAM version $Revision: 5.10 $                                                                                                                                                                                                           
-------------------------------------------------------------                                                                                                                                                                              
This system uses 8 bytes per array element.                                                                                                                                                                                                
-------------------------------------------------------------                                                                                                                                                                              
Array size = 10000000 (elements), Offset = 0 (elements)                                                                                                                                                                                    
Memory per array = 76.3 MiB (= 0.1 GiB).                                                                                                                                                                                                   
Total memory required = 228.9 MiB (= 0.2 GiB).                                                                                                                                                                                             
Each kernel will be executed 10 times.                                                                                                                                                                                                     
 The *best* time for each kernel (excluding the first iteration)                                                                                                                                                                           
 will be used to compute the reported bandwidth.                                                                                                                                                                                           
-------------------------------------------------------------                                                                                                                                                                              
Your clock granularity/precision appears to be 1 microseconds.                                                                                                                                                                             
Each test below will take on the order of 25614 microseconds.                                                                                                                                                                              
   (= 25614 clock ticks)                                                                                                                                                                                                                   
Increase the size of the arrays if this shows that                                                                                                                                                                                         
you are not getting at least 20 clock ticks per test.                                                                                                                                                                                      
-------------------------------------------------------------                                                                                                                                                                              
WARNING -- The above is only a rough guideline.                                                                                                                                                                                            
For best results, please be sure you know the                                                                                                                                                                                              
precision of your system timer.                                                                                                                                                                                                            
-------------------------------------------------------------                                                                                                                                                                              
Function    Best Rate MB/s  Avg time     Min time     Max time                                                                                                                                                                             
Copy:            7981.3     0.020670     0.020047     0.022545                                                                                                                                                                             
Scale:           9484.8     0.018009     0.016869     0.021216                                                                                                                                                                             
Add:            10859.3     0.022774     0.022101     0.024963                                                                                                                                                                             
Triad:          11370.7     0.022250     0.021107     0.024320                                                                                                                                                                             
-------------------------------------------------------------                                                                                                                                                                              
Solution Validates: avg error less than 1.000000e-13 on all three arrays                                                                                                                                                                   
-------------------------------------------------------------                                                                                                                                                                              
                                                                                

[codecov]

Codecov Report

Base: 73.22% // Head: 72.64% // Decreases project coverage by -0.58% ⚠️

Coverage data is based on head (eb7f39a) compared to base (20c9354).
Patch coverage: 1.96% of modified lines in pull request are covered.

❗ Current head eb7f39a differs from pull request most recent head 904b8f5. Consider uploading reports for the commit 904b8f5 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##            devel    #1313      +/-   ##
==========================================
- Coverage   73.22%   72.64%   -0.58%     
==========================================
  Files          56       56              
  Lines        5923     5972      +49     
  Branches     1051     1058       +7     
==========================================
+ Hits         4337     4338       +1     
- Misses       1584     1632      +48     
  Partials        2        2              

Impacted Files	Coverage Δ
buildtest/builders/base.py	51.15% <1.96%> (-4.66%)	⬇️
buildtest/tools/stylecheck.py	89.83% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.