-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix intersection-typed function calls with union-typed arguments #514
Fix intersection-typed function calls with union-typed arguments #514
Conversation
…nion_arg_should_pass.erl And rearrange some already existing tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's solved by this PR? No new should pass or should fail, only some changes in known problems? Did you forget to git add them?
This means that we might have to forego the nicety of preserving (non)emptiness of lists through function calls :'(
I don't understand why. An empty list and a non-empty list are not overlapping. If a function f([]) -> []; ([A, ...]) -> [A, ...]
is called with a non-empty or empty list, it's type can be preserved, can't it?
test/known_problems/should_pass/call_intersection_function_with_union_arg_should_pass.erl
Outdated
Show resolved
Hide resolved
This PR is still a draft as you can see. Things are not moved to their intended destinations yet, although they are in the repo. I'll be cleaning it up. What works now, but didn't, is for example https://github.com/erszcz/Gradualizer/blob/ed424bac9657dd2048efb5aaeb056dc66df270c3/test/known_problems/should_pass/call_intersection_function_with_union_arg_should_pass.erl. On master:
This PR:
But a %% Preserve the (non)empty property of the input list.
-spec map(fun((A) -> B), [A, ...]) -> [B, ...];
(fun((A) -> B), [A]) -> [B]. which is sad, as they're very convenient. It's often the case that we want to take |
Specifically, calling intersection-typed functions with union-typed arguments.
…on_with_union_arg_{should_ ->}pass.erl
Overlapping clauses lead to an unsolvable problem when selecting which clause of an intersection-typed function to call.
Strictly speaking, they're not that accurate anymore, since any list containing even a single binary will lead to returning the result as a binary. It's not reflected in the current specs.
ed424ba
to
b4b008b
Compare
Quick summary of where we're at. The following: -spec i2(t(), u()) -> one | two.
i2(T, U) ->
j(T, U).
-spec j(t1, u1) -> one;
(t2, u2) -> two.
j(t1, u1) -> one;
j(t2, u2) -> two. doesn't pass anymore - it's properly detected to be an error ✅ We're detecting overlapping spec clauses when functions with such are called (just a printout right now, but can be thrown as a proper error) ✅ The price to pay for the above is slowness... big enough to significantly slow down the self-check ❌ Moreover, there's a relatively high number of self-check errors on this branch, too (around 200 lines) ❌ Another loss is that: -spec i(a, b) -> {a, b};
(d, e) -> {d, e}.
i(V, U) -> {V, U}.
-spec j({a, b} | {d, e}) -> {a, b} | {d, e}.
j({V, U}) -> i(V, U). does NOT type check anymore ❌ Next steps:
|
Some new results:
It's not as simple as I thought. The condition that we detect doesn't necessarily mean that the called function has an overlapping spec - that's only one of the possibilities, the others being less defined. I think we have to stay with the more general
I've run The code needs cleanup, of course, but things don't look bad as of now. |
7b11ac8
to
e872104
Compare
Ahh, it's not just overlapping spec clauses that cause problems. The problem is that we get multiple clauses upon selection based on argument types - overlapping specs might lead to this, but it might also happen if argument types are any of the "gradual" types, i.e. If we allow such gradual types to match multiple spec clauses, then we can just as well allow overlapping spec clauses. This means we have to take the LUB of the clauses' return types as the return type of the call. In the light of overlapping specs, it's useless, though, since for a spec like: -spec map(fun((A) -> B), [A, ...]) -> [B, ...];
(fun((A) -> B), [A]) -> [B]. We would still need to deal with a call result type like hd(lists:map(F, NonEmptyList)) I think it's fair to say that "the less we know, the less we can guarantee", and therefore where it's possible to return a single clause's return type, but where multiple clauses match the call argument types return the union of these clauses' return types. This way the more precise types the programmer provides, the more specific feedback Gradualizer will return. |
c0206bb
to
bf32ae0
Compare
Ok, I think this PR is ready functionally-wise. I'll clean up the code and git history a bit and then switch it to "Ready for review". |
b6244bd
to
f4f78ce
Compare
This allows arg types to select multiple matching spec clauses and uses a LUB of the result types as the call result type.
…th_union_arg_fail
…ing erlang:'++'/2 spec
…ion_with_union_arg_pass.erl
…uple_union_arg.erl
f4f78ce
to
792d13f
Compare
Ok, I think this is finally ready. @zuiderkwast WDYT? |
If there are no review comments I'll be merging this shortly |
I think it'd be worth to do some kind of impact assessment of not supporting overlapping specs. How common are they in the wild? Does OTP use them? My initial philosophy for Gradualizer was to go to great lengths to support the kind of code that people actually write. It will still be possible to write overlapping specs. What's your plan for what Gradualizer should do with them? To be clear, I'm not necessarily against this change. But it feels like quite a big step to rule out perfectly well-formed specs, just because they are overlapping. So it'd be good think about what the impact will be and ensure that the behaviour of Gradualizer is not too surprising, should someone write an overlapping spec by accident, say. |
I think these specs aren't common at all. Erlang docs warn about overlapping specs not being supported by Dialyzer:
I think all (or at least most?) of them in the tests and Gradualizer source code were introduced by me after #461, when it turned out they can be useful for preserving some properties "through" a function call, but it was not clear yet that function call checking in general doesn't handle union typed args to intersection typed functions properly.
Strictly speaking, this PR doesn't disallow them, they're still handled as best as possible, i.e. if a function spec has overlapping clauses, the result type of the function call is assumed to be the union of clauses' result types. However, in practice this usually means we lose the ability to preserve non-emptiness of a list through the call in a case like -spec map(fun((A) -> B), [A, ...]) -> [B, ...];
(fun((A) -> B), [A]) -> [B]. because, for example, a non-empty list of Ultimately, though, I think we have to pay this price for being able to properly check that the call to -spec f(a | b) -> a | b.
f(V) ->
h(V).
-spec h(a) -> a;
(b) -> b.
h(V) -> V. In other words, we have to check that all clauses of a multi-clause spec, in total, cover a union-typed argument, not that each of the spec clauses does it. The above example fails on master, but does not fail with this PR. Otherwise we'll be swamped with false positives about perfectly valid code. |
Breaking something to fix something else is what I was worried about too. I think it's fine to drop overlapping specs, but what about gradually-overlapping? We use -spec f(integer()) -> apa;
(any()) -> bepa. ... and trust that this particular instance of |
Breaking something that was introduced recently as an experiment to fix something that's fundamental, but never worked properly, seems like a good enough reason to me ;) @zuiderkwast I think that to be on the safe side we have to assume that |
Yes I agree, let's merge this.
|
In the paper about gradual typing with union-intersection-negation types, they rewrite the spec until it's not overlapping anymore. The above would be rewritten to |
Yes, this could be a good way forward. It introduces an ordering of the type declarations, so it matters in what order they are written. But it's a fairly small price to pay for the extra expressivity. The current situation of allowing people to write overlapping types but discouraging it and not handling it well is not a good status quo. It's very easy to accidentally write overlapping types and, as you point out @zuiderkwast, it becames even easier when you involve gradual types. |
This version brings approx. 30 new PRs. The highlights are: - Improve map exhaustiveness checking [erlang-ls#524](josefs/Gradualizer#524) by @xxdavid - Fix all remaining self-check errors [erlang-ls#521](josefs/Gradualizer#521) by @erszcz - Fix intersection-typed function calls with union-typed arguments [erlang-ls#514](josefs/Gradualizer#514) by @erszcz - Experimental constraint solver [erlang-ls#450](josefs/Gradualizer#450) by @erszcz
This version brings approx. 30 PRs. The highlights are: - Improve map exhaustiveness checking [erlang-ls#524](josefs/Gradualizer#524) by @xxdavid - Fix all remaining self-check errors [erlang-ls#521](josefs/Gradualizer#521) by @erszcz - Fix intersection-typed function calls with union-typed arguments [erlang-ls#514](josefs/Gradualizer#514) by @erszcz - Experimental constraint solver [erlang-ls#450](josefs/Gradualizer#450) by @erszcz
This version brings approx. 30 PRs. The highlights are: - Improve map exhaustiveness checking [erlang-ls#524](josefs/Gradualizer#524) by @xxdavid - Fix all remaining self-check errors [erlang-ls#521](josefs/Gradualizer#521) by @erszcz - Fix intersection-typed function calls with union-typed arguments [erlang-ls#514](josefs/Gradualizer#514) by @erszcz - Experimental constraint solver [erlang-ls#450](josefs/Gradualizer#450) by @erszcz
This version brings approx. 30 PRs. The highlights are: - Improve map exhaustiveness checking [#524](josefs/Gradualizer#524) by @xxdavid - Fix all remaining self-check errors [#521](josefs/Gradualizer#521) by @erszcz - Fix intersection-typed function calls with union-typed arguments [#514](josefs/Gradualizer#514) by @erszcz - Experimental constraint solver [#450](josefs/Gradualizer#450) by @erszcz
This PR extracts the intersection-typed function call logic improvements from #512 to offer a smaller chunk of code in a single PR. Without this functionality in place it will be very hard to finish #512 - that is fixing the new self-check errors there.
The good news is that this PR fixes a number of tests (including some new ones) and solves the problem of calling a function with an intersection type with an argument with a union type.
The bad news is that this PR makes it obvious why overlapping spec clauses, mentioned in #461, are bad. They make it impossible to tell which clause's result type is the correct result type of the call. I'm not sure how to solve this yet or if it's possible at all. This means that we might have to forego the nicety of preserving (non)emptiness of lists through function calls :'(
Constraints aren't handled properly yet, so some tests fail because of that.