-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
50 add performance tests #57
Conversation
No CI yet. #50
Coverage Report
|
Coverage Report
|
Coverage Report
|
Coverage Report
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Benchmark
Benchmark suite | Current: 6c0e239 | Previous: 90558b6 | Ratio |
---|---|---|---|
edtf/natlang/tests.py::test_benchmark_natlang[23rd Dynasty-None] |
70641.72248747271 iter/sec (stddev: 7.631740109706085e-7 ) |
117838.09411600193 iter/sec (stddev: 9.140748993217583e-7 ) |
1.67 |
edtf/natlang/tests.py::test_benchmark_natlang[January 2008-2008-01] |
8265.027515352316 iter/sec (stddev: 0.00000624553173262339 ) |
11449.295346157909 iter/sec (stddev: 0.00000546797602185004 ) |
1.39 |
edtf/natlang/tests.py::test_benchmark_natlang[ca1860-1860~] |
9946.622189510234 iter/sec (stddev: 0.0000060038117620762754 ) |
14018.980954980647 iter/sec (stddev: 0.000004190733350336885 ) |
1.41 |
edtf/natlang/tests.py::test_benchmark_natlang[uncertain: approx 1862-1862%] |
6767.269493555209 iter/sec (stddev: 0.000006786669661079336 ) |
9377.335229826682 iter/sec (stddev: 0.000005341933165905862 ) |
1.39 |
edtf/natlang/tests.py::test_benchmark_natlang[January-XXXX-01] |
11463.57222196724 iter/sec (stddev: 0.000009888909809905773 ) |
16635.89085978395 iter/sec (stddev: 0.000003722896762661871 ) |
1.45 |
edtf/natlang/tests.py::test_benchmark_natlang[Winter 1872-1872-24] |
8494.477221134308 iter/sec (stddev: 0.000006319040682758152 ) |
11799.981255815099 iter/sec (stddev: 0.000004886697630423069 ) |
1.39 |
edtf/natlang/tests.py::test_benchmark_natlang[before approx January 18 1928-/1928-01-18~] |
5693.466862411181 iter/sec (stddev: 0.0000074192037469237325 ) |
7327.001686715526 iter/sec (stddev: 0.000014874586617515084 ) |
1.29 |
edtf/natlang/tests.py::test_benchmark_natlang[birthday in 1872-1872] |
7345.582820224108 iter/sec (stddev: 0.000007711477707651927 ) |
10113.290756146082 iter/sec (stddev: 0.000004521646195592372 ) |
1.38 |
edtf/natlang/tests.py::test_benchmark_natlang[1270 CE-1270] |
49730.52522186321 iter/sec (stddev: 0.0000014177870959215635 ) |
69415.36587515069 iter/sec (stddev: 0.000001258463340756581 ) |
1.40 |
edtf/natlang/tests.py::test_benchmark_natlang[2nd century bce--01XX] |
44622.733371628405 iter/sec (stddev: 0.0000013189163197732718 ) |
57149.619901531514 iter/sec (stddev: 0.0000010163156920467694 ) |
1.28 |
edtf/natlang/tests.py::test_benchmark_natlang[1858/1860-[1858, 1860]] |
22585.42489715447 iter/sec (stddev: 0.00000290588232244421 ) |
32295.71927034893 iter/sec (stddev: 0.000003964388925568352 ) |
1.43 |
edtf/parser/tests.py::test_benchmark_parser[2001-02-03] |
127.11360552437587 iter/sec (stddev: 0.0005263511089472479 ) |
147.90762624443573 iter/sec (stddev: 0.000677037340647177 ) |
1.16 |
edtf/parser/tests.py::test_benchmark_parser[2008-12] |
134.09690138556914 iter/sec (stddev: 0.0015129888460220502 ) |
154.10346486830713 iter/sec (stddev: 0.002119069171598076 ) |
1.15 |
edtf/parser/tests.py::test_benchmark_parser[2008] |
162.22746462824088 iter/sec (stddev: 0.0020802469862917193 ) |
193.72321366345068 iter/sec (stddev: 0.0015694673864413153 ) |
1.19 |
edtf/parser/tests.py::test_benchmark_parser[-0999] |
168.8424534076089 iter/sec (stddev: 0.001335171793366204 ) |
192.4055135031146 iter/sec (stddev: 0.0021115296335792004 ) |
1.14 |
edtf/parser/tests.py::test_benchmark_parser[2004-01-01T10:10:10+05:00] |
112.52475921460926 iter/sec (stddev: 0.0020055908064093406 ) |
132.30107724950307 iter/sec (stddev: 0.0022513662265232224 ) |
1.18 |
edtf/parser/tests.py::test_benchmark_parser[-2005/-1999-02] |
101.60889128215474 iter/sec (stddev: 0.0018984994313346053 ) |
117.33029498812579 iter/sec (stddev: 0.0024572631034443 ) |
1.15 |
edtf/parser/tests.py::test_benchmark_parser[/2006] |
211.62459821791225 iter/sec (stddev: 0.0011246478706547632 ) |
261.5659554104642 iter/sec (stddev: 0.00012280976585137294 ) |
1.24 |
edtf/parser/tests.py::test_benchmark_parser[?2004-%06] |
175.12035346040562 iter/sec (stddev: 0.00020742709140351162 ) |
204.48382538973428 iter/sec (stddev: 0.0011968975778576395 ) |
1.17 |
edtf/parser/tests.py::test_benchmark_parser[[1667, 1760-12]] |
18.699448530262654 iter/sec (stddev: 0.0033446667504534455 ) |
22.079815197887676 iter/sec (stddev: 0.0004880141774492468 ) |
1.18 |
edtf/parser/tests.py::test_benchmark_parser[Y3388E2S3] |
393.6419880843278 iter/sec (stddev: 0.00008406965657298834 ) |
496.9792544677667 iter/sec (stddev: 0.00008549226261985391 ) |
1.26 |
edtf/parser/tests.py::test_benchmark_parser[2001-29] |
96.59289844583233 iter/sec (stddev: 0.001866637366794991 ) |
112.09680919584098 iter/sec (stddev: 0.0022428458952360266 ) |
1.16 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is great! That Action seemed fairly easy to integrate, as does pytest-benchmark
. It seems like the natlang
parser is very fast, with the EDTF parser running much slower. It was pretty obvious on some of those tests when you enabled packrat
, nice performance bump there. Not sure what some of the other oscillations are - perhaps just variance in runs?
@pytest.mark.benchmark | ||
@pytest.mark.parametrize("test_input", BENCHMARK_EXAMPLES) | ||
def test_benchmark_parser(benchmark, test_input): | ||
"""Benchmark parsing of selected EDTF strings.""" | ||
benchmark(parse, test_input) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems very smooth to integrate this
#50 Add benchmark tests, publish results in CI, activate packrat.
Results will appear at https://ixc.github.io/python-edtf/dev/bench/ as well as in comments on the commit.