-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a Hypothesis plugin #2097
Add a Hypothesis plugin #2097
Conversation
55517f8
to
e7a11a5
Compare
Codecov Report
@@ Coverage Diff @@
## master #2097 +/- ##
===========================================
- Coverage 100.00% 99.88% -0.12%
===========================================
Files 21 22 +1
Lines 4202 4323 +121
Branches 855 873 +18
===========================================
+ Hits 4202 4318 +116
- Misses 0 5 +5
Continue to review full report at Codecov.
|
c54f1ba
to
8232803
Compare
Amazing work @Zac-HD! So glad to see this PR open! |
8232803
to
318bc56
Compare
@PrettyWood - thanks! I'm looking forward to seeing what people do with it 😃 There was one deliberately unreachable line (assertion that we returned from a loop body), which I've unrolled so we get 100% coverage without needing lots of pragmas. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a few small things, overall I think this is looking great.
b835b9d
to
1b10c1e
Compare
OK @samuelcolvin - I think I'm done, and have reverted the #2155 patch in favor of a test-only solution 😄 |
@Zac-HD I was wondering, would it be possible to add generic support for constrained types like st.register_type_strategy(
ConstrainedStr,
lambda T: st.from_regex(T.regex, fullmatch=True) if T.regex is not None else st.text(min_size=T.min_length, max_size=T.max_length)
) Unfortunately, that type strategy is not picked up on for subclasses of class MyString(ConstrainedStr):
regex = "[A-Z]{2,8}"
st.from_type(MyString).example() # Returns empty string This doesn't need to hold back this PR, but it is related so I thought I'd post it here. |
I was all ready to explain why we couldn't, but in fact there is a way... it's just a little more complicated than you might expect. First, the obvious approach of registering So the trick is to register a strategy for each child class when that class is created, which can be done from the I'll update this PR in a week or two, or if you'd like to merge it sooner I can open a follow-up instead 😁 |
0d5f569
to
eb7aaca
Compare
81efff5
to
4e193cf
Compare
Great job again @Zac-HD! I'll probably use it as soon as v1.8 is released 🚀 |
4e193cf
to
134942d
Compare
134942d
to
af5c8e3
Compare
Thanks for the review @PrettyWood! I've added your suggestions, along with even more comments to explain what's happening 😄 |
032870b
to
af5c8e3
Compare
this is awesome, thank you so much! 🚀 🙏 🥳 I'm working through PRs now, v1.8 coming soon. |
Woohoo! Pydantic is also awesome, I'm so excited about combining it with Hypothesis 😁 Can't wait to see what people do with this either |
This patch adds a plugin which teaches Hypothesis how to generated examples of Pydantic's custom field types (which closes #2017). Note that there is no runtime impact or dependency; this module is imported by Hypothesis so it only exists at test-time and only when Hypothesis is both installed and actually being used.
changes/<pull request or issue id>-<github username>.md
file added describing change(see changes/README.md for details)
I've ended up dropping the integration for URLs, and for constrained lists and sets... but everything else is ready to go!
Why not support constrained lists and sets?
In short, I couldn't get them to play nicely with Hypothesis' type registry - because the runtime objects are subclasses of a parametrized generic type, and aren't always distinguished by our introspection logic 😭
Why I decided to leave our URLs (with code)
I spent a long time tweaking support for URLs, but ultimately gave it up: it's easy to generate valid URLs, unless you want to generally really strange ones which will find lots of bugs. I decided that it was better to make users register their own - and make the tradeoff explicit - than to cover up the complexity and have automatic but weak tests. This is a standard design choice for Hypothesis, sadly, so I've included my code so far below for the benefit of any future contributor who wants to pick it up.
`Literal[None]` requires Python 3.7+ or a recent version of Hypothesis
Because I literally just fixed that this evening - since it requires a backport on a security-only Python version, we delayed working out how to support it until we knew it would actually be used before the 3.6 EOL later this year.