Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move test framework tests to Cadence #2129

Merged
merged 1 commit into from
Nov 10, 2022
Merged

Conversation

turbolent
Copy link
Member

@turbolent turbolent commented Nov 9, 2022

Work towards #2084

Description

When updating the Test framework, cadence-tools/test, to Stable Cadence, I realized I had broken it in #2125.
This was not catched, as all tests for the standard library test contract are in the downstream repo.

To avoid future breakage, move some of the functionality and tests to the Cadence repo.


  • Targeted PR against master branch
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work
  • Code follows the standards mentioned here
  • Updated relevant documentation
  • Re-reviewed Files changed in the Github PR explorer
  • Added appropriate labels

@codecov
Copy link

codecov bot commented Nov 9, 2022

Codecov Report

Merging #2129 (b4dfd5b) into master (15bbb25) will increase coverage by 0.19%.
The diff coverage is 37.50%.

@@            Coverage Diff             @@
##           master    #2129      +/-   ##
==========================================
+ Coverage   77.75%   77.94%   +0.19%     
==========================================
  Files         305      305              
  Lines       63629    63676      +47     
==========================================
+ Hits        49476    49634     +158     
+ Misses      12383    12256     -127     
- Partials     1770     1786      +16     
Flag Coverage Δ
unittests 77.94% <37.50%> (+0.19%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
runtime/stdlib/test.go 38.19% <37.50%> (+16.10%) ⬆️
runtime/sema/type.go 89.96% <0.00%> (+0.08%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@github-actions
Copy link

github-actions bot commented Nov 9, 2022

Cadence Benchstat comparison

This branch with compared with the base branch onflow:master commit 15bbb25
The command for i in {1..N}; do go test ./... -run=XXX -bench=. -benchmem -shuffle=on; done was used.
Bench tests were run a total of 7 times on each branch.

Collapsed results for better readability

old.txtnew.txt
time/opdelta
CheckContractInterfaceFungibleTokenConformance-2159µs ± 7%154µs ± 8%~(p=0.165 n=7+7)
ContractInterfaceFungibleToken-251.3µs ± 6%50.4µs ±13%~(p=0.259 n=7+7)
InterpretRecursionFib-23.08ms ± 5%3.03ms ±13%~(p=0.318 n=7+7)
NewInterpreter/new_interpreter-21.39µs ±11%1.37µs ± 8%~(p=0.620 n=7+7)
NewInterpreter/new_sub-interpreter-2736ns ± 4%709ns ± 6%~(p=0.073 n=7+7)
ParseArray-29.90ms ± 3%9.59ms ± 7%~(p=0.128 n=7+7)
ParseDeploy/byte_array-215.0ms ±17%14.5ms ± 3%~(p=0.731 n=7+6)
ParseDeploy/decode_hex-21.54ms ± 4%1.50ms ± 4%~(p=0.209 n=7+7)
ParseFungibleToken/With_memory_metering-2244µs ± 4%232µs ± 7%~(p=0.073 n=7+7)
ParseFungibleToken/Without_memory_metering-2192µs ± 2%183µs ± 6%~(p=0.051 n=6+7)
ParseInfix-28.63µs ± 6%8.37µs ± 7%~(p=0.383 n=7+7)
QualifiedIdentifierCreation/One_level-23.00ns ± 4%2.93ns ± 6%~(p=0.209 n=7+7)
QualifiedIdentifierCreation/Three_levels-2177ns ± 5%172ns ± 5%~(p=0.091 n=7+7)
RuntimeFungibleTokenTransfer-2824µs ±21%856µs ±25%~(p=1.000 n=7+7)
RuntimeResourceDictionaryValues-26.46ms ± 6%6.38ms ±10%~(p=0.620 n=7+7)
RuntimeScriptNoop-228.9µs ±36%24.9µs ±29%~(p=0.209 n=7+7)
SuperTypeInference/arrays-2395ns ± 3%390ns ± 7%~(p=0.383 n=7+7)
SuperTypeInference/composites-2153ns ± 3%146ns ± 5%−4.40%(p=0.026 n=7+7)
SuperTypeInference/integers-2115ns ± 6%112ns ± 6%~(p=0.364 n=7+7)
ValueIsSubtypeOfSemaType-2119ns ± 4%118ns ± 3%~(p=0.310 n=6+6)
 
alloc/opdelta
CheckContractInterfaceFungibleTokenConformance-250.1kB ± 0%50.1kB ± 0%~(all equal)
ContractInterfaceFungibleToken-224.2kB ± 0%24.2kB ± 0%~(p=1.000 n=7+7)
InterpretRecursionFib-21.00MB ± 0%1.00MB ± 0%~(p=0.462 n=7+7)
NewInterpreter/new_interpreter-2752B ± 0%752B ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-2200B ± 0%200B ± 0%~(all equal)
ParseArray-22.76MB ± 4%2.76MB ± 6%~(p=0.805 n=7+7)
ParseDeploy/byte_array-24.10MB ± 0%4.26MB ± 4%+3.79%(p=0.034 n=6+7)
ParseDeploy/decode_hex-2214kB ± 0%214kB ± 0%~(p=0.446 n=7+7)
ParseFungibleToken/With_memory_metering-229.4kB ± 0%29.4kB ± 0%~(p=0.776 n=7+7)
ParseFungibleToken/Without_memory_metering-229.4kB ± 0%29.4kB ± 0%~(p=0.200 n=7+7)
ParseInfix-21.93kB ± 0%1.93kB ± 0%~(p=0.840 n=7+7)
QualifiedIdentifierCreation/One_level-20.00B 0.00B ~(all equal)
QualifiedIdentifierCreation/Three_levels-264.0B ± 0%64.0B ± 0%~(all equal)
RuntimeFungibleTokenTransfer-2103kB ± 1%103kB ± 1%~(p=0.710 n=7+7)
RuntimeResourceDictionaryValues-22.28MB ± 0%2.28MB ± 0%~(p=0.366 n=6+7)
RuntimeScriptNoop-28.35kB ± 0%8.35kB ± 0%~(p=0.836 n=6+7)
SuperTypeInference/arrays-296.0B ± 0%96.0B ± 0%~(all equal)
SuperTypeInference/composites-20.00B 0.00B ~(all equal)
SuperTypeInference/integers-20.00B 0.00B ~(all equal)
ValueIsSubtypeOfSemaType-248.0B ± 0%48.0B ± 0%~(all equal)
 
allocs/opdelta
CheckContractInterfaceFungibleTokenConformance-2900 ± 0%900 ± 0%~(all equal)
ContractInterfaceFungibleToken-2434 ± 0%434 ± 0%~(all equal)
InterpretRecursionFib-218.9k ± 0%18.9k ± 0%~(all equal)
NewInterpreter/new_interpreter-213.0 ± 0%13.0 ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-24.00 ± 0%4.00 ± 0%~(all equal)
ParseArray-269.6k ± 0%69.6k ± 0%~(p=0.098 n=7+6)
ParseDeploy/byte_array-2104k ± 0%104k ± 0%−0.03%(p=0.042 n=7+6)
ParseDeploy/decode_hex-275.0 ± 0%75.0 ± 0%~(all equal)
ParseFungibleToken/With_memory_metering-20.99k ± 0%0.99k ± 0%~(all equal)
ParseFungibleToken/Without_memory_metering-20.99k ± 0%0.99k ± 0%~(all equal)
ParseInfix-260.0 ± 0%60.0 ± 0%~(all equal)
QualifiedIdentifierCreation/One_level-20.00 0.00 ~(all equal)
QualifiedIdentifierCreation/Three_levels-22.00 ± 0%2.00 ± 0%~(all equal)
RuntimeFungibleTokenTransfer-22.06k ± 0%2.06k ± 0%~(p=0.192 n=7+7)
RuntimeResourceDictionaryValues-237.0k ± 0%37.0k ± 0%~(p=0.274 n=7+7)
RuntimeScriptNoop-2138 ± 0%138 ± 0%~(all equal)
SuperTypeInference/arrays-23.00 ± 0%3.00 ± 0%~(all equal)
SuperTypeInference/composites-20.00 0.00 ~(all equal)
SuperTypeInference/integers-20.00 0.00 ~(all equal)
ValueIsSubtypeOfSemaType-21.00 ± 0%1.00 ± 0%~(all equal)
 

@turbolent turbolent mentioned this pull request Nov 9, 2022
6 tasks
@turbolent turbolent merged commit e3a42e1 into master Nov 10, 2022
@turbolent turbolent deleted the bastian/test-test-contract branch November 10, 2022 00:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants