limit recursion depth to prevent stack overflow in 2-arg `promote_type` #57517

nsajko · 2025-02-24T11:37:39Z

Limit recursion depth to prevent stack overflows.

adienes · 2025-02-24T19:21:23Z

what is the advantage over #57507 ? this seems to add quite a bit more complexity and includes scary things like hardcoded constants

nsajko · 2025-02-24T19:32:37Z

what is the advantage over #57507 ?

This PR prevents a much wider class of stack overflows. In particular, #57507 doesn't fix #13193 (see the test here for an example), while this PR should.

scary things like hardcoded constants

Usually I avoid hardcoded constants as best as I can, but in this case there's no way to guarantee (local) termination without hardcoding a cutoff. Another approach would be to stick with recursion, but limit the depth, however in any case it definitely seems necessary to have some kind of hardcoded limit.

promote_type shouldn't run without bound just because of faulty/conflicting promote_rules. Or, worse, overflow the stack. Or, even worse, cause the compiler itself to run without bound while it's trying to compile the promote_type method.

nsajko · 2025-02-25T06:28:32Z

This is how the behavior for #13193 looks with this PR, due to the limited recursion depth:

julia> struct SIQuantity{T<:Number} <: Number; end

julia> Base.promote_rule(::Type{SIQuantity{T}}, ::Type{SIQuantity{S}}) where {T, S} = SIQuantity{promote_type(T,S)}

julia> Base.promote_rule(::Type{SIQuantity{T}}, ::Type{S}) where {T, S<:Number} = SIQuantity{promote_type(T,S)}

julia> struct Interval{T<:Number} <: Number; end

julia> Base.promote_rule(::Type{Interval{T}}, ::Type{Interval{S}}) where {T, S} = Interval{promote_type(T,S)}

julia> Base.promote_rule(::Type{Interval{T}}, ::Type{S}) where {T, S<:Number} = Interval{promote_type(T,S)}

julia> promote_type(Interval{Int}, SIQuantity{Int})
ERROR: ArgumentError: `promote_type`: recursion depth limit reached, giving up; check for faulty/conflicting/missing `promote_rule` methods
Stacktrace:
  [1] _promote_type_binary(::Type, ::Type, ::Tuple{})
    @ Base ./promotion.jl:308
  [2] promote_result(::Type, ::Type, ::Type{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{Int64}}}}}}}}}}}}, ::Type{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Int64}}}}}}}}}}}}, l::Tuple{})
    @ Base ./promotion.jl:360
  [3] _promote_type_binary(::Type{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Int64}}}}}}}}}}}, ::Type{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{Int64}}}}}}}}}}}, recursion_depth_limit::Tuple{Nothing})
    @ Base ./promotion.jl:325
  [4] promote_result(::Type, ::Type, ::Type{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Int64}}}}}}}}}}}, ::Type{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{Int64}}}}}}}}}}}, l::Tuple{Nothing})
    @ Base ./promotion.jl:360
  [5] _promote_type_binary(::Type{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{Int64}}}}}}}}}}, ::Type{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Int64}}}}}}}}}}, recursion_depth_limit::Tuple{Nothing, Nothing})
    @ Base ./promotion.jl:325
  [6] promote_result(::Type, ::Type, ::Type{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{Int64}}}}}}}}}}, ::Type{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Int64}}}}}}}}}}, l::Tuple{Nothing, Nothing})
    @ Base ./promotion.jl:360
  [7] _promote_type_binary
    @ ./promotion.jl:325 [inlined]
  [8] promote_result(::Type, ::Type, ::Type{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Int64}}}}}}}}}, ::Type{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{Int64}}}}}}}}}, l::Tuple{Nothing, Nothing, Nothing})
    @ Base ./promotion.jl:360
  [9] _promote_type_binary
    @ ./promotion.jl:325 [inlined]
 [10] promote_result(::Type, ::Type, ::Type{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{Int64}}}}}}}}, ::Type{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Int64}}}}}}}}, l::NTuple{4, Nothing})
    @ Base ./promotion.jl:360
 [11] _promote_type_binary
    @ ./promotion.jl:325 [inlined]
 [12] promote_result(::Type, ::Type, ::Type{Interval{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Int64}}}}}}}, ::Type{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Interval{Int64}}}}}}}, l::NTuple{5, Nothing})
    @ Base ./promotion.jl:360
 [13] _promote_type_binary
    @ ./promotion.jl:325 [inlined]
 [14] promote_result(::Type, ::Type, ::Type{Interval{SIQuantity{Interval{SIQuantity{Interval{Int64}}}}}}, ::Type{SIQuantity{Interval{SIQuantity{Interval{SIQuantity{Int64}}}}}}, l::NTuple{6, Nothing})
    @ Base ./promotion.jl:360
 [15] _promote_type_binary
    @ ./promotion.jl:325 [inlined]
 [16] promote_result(::Type, ::Type, ::Type{Interval{SIQuantity{Interval{SIQuantity{Int64}}}}}, ::Type{SIQuantity{Interval{SIQuantity{Interval{Int64}}}}}, l::NTuple{7, Nothing})
    @ Base ./promotion.jl:360
 [17] _promote_type_binary
    @ ./promotion.jl:325 [inlined]
 [18] promote_result(::Type, ::Type, ::Type{Interval{SIQuantity{Interval{Int64}}}}, ::Type{SIQuantity{Interval{SIQuantity{Int64}}}}, l::NTuple{8, Nothing})
    @ Base ./promotion.jl:360
 [19] _promote_type_binary
    @ ./promotion.jl:325 [inlined]
 [20] promote_result(::Type, ::Type, ::Type{Interval{SIQuantity{Int64}}}, ::Type{SIQuantity{Interval{Int64}}}, l::NTuple{9, Nothing})
    @ Base ./promotion.jl:360
 [21] _promote_type_binary
    @ ./promotion.jl:325 [inlined]
 [22] promote_type(::Type{Interval{Int64}}, ::Type{SIQuantity{Int64}})
    @ Base ./promotion.jl:340
 [23] top-level scope
    @ REPL[7]:1

base/promotion.jl

adienes · 2025-02-25T13:36:26Z

not sure if I'm 100% qualified to review beyond stylistic things but as far as I can tell I like this approach

it might make sense to run the benchmarks? I know this shouldn't change anything but promote_type is quite a highly used function so just in case

nsajko · 2025-02-25T21:42:34Z

Question for triage: does limiting the recursion depth seem like an acceptable minor change? It's a bugfix, as it prevents stack overflows, but hypothetically some user code could break if it relied on an awkward chain of promote_rules.

NB: an earlier version of this PR tried to do away with recursion altogether in favor of a loop, however that resulted in inference regressions.

NB: setting the recursion depth limit higher breaks bootstrap for some reason, so currently it's not possible to set the limit higher.

Don't use recursion, delete the `promote_result` function. Use loop known to terminate, because of a hardcoded limit on the iteration count. Closes JuliaLang#57507, this PR encompasses the fixes from that PR and is more comprehensive. Fixes JuliaLang#13193

nsajko · 2025-02-26T20:25:26Z

@nanosoldier runtests()

nsajko added the bugfix This change fixes an existing bug label Feb 24, 2025

nsajko mentioned this pull request Feb 24, 2025

Informative error in promotion with conflicting promote_rules #57507

Open

nsajko force-pushed the prevent_stack_overflow_in_type_promotion branch from 57dbdeb to 69e5d32 Compare February 24, 2025 14:31

nsajko force-pushed the prevent_stack_overflow_in_type_promotion branch from 9e1f5dc to 4b7bb02 Compare February 24, 2025 22:00

nsajko added the backport 1.12 Change should be backported to release-1.12 label Feb 25, 2025

nsajko added error handling Handling of exceptions by Julia or the user error messages Better, more actionable error messages labels Feb 25, 2025

nsajko requested review from aviatesk and adienes February 25, 2025 06:48

adienes reviewed Feb 25, 2025

View reviewed changes

base/promotion.jl Show resolved Hide resolved

nsajko added minor change Marginal behavior change acceptable for a minor release needs pkgeval Tests for all registered packages should be run with this change needs nanosoldier run This PR should have benchmarks run on it labels Feb 25, 2025

nsajko changed the title ~~prevent stack overflow in 2-arg promote_type~~ limit recursion depth to prevent stack overflow in 2-arg promote_type Feb 25, 2025

nsajko added triage This should be discussed on a triage call and removed backport 1.12 Change should be backported to release-1.12 labels Feb 25, 2025

nsajko added 9 commits February 26, 2025 21:24

use types_are_identical where intended

f2d0659

fix: use== instead of ===

d11d136

rename function

012016f

fix bootstrap

c6f7969

use similar implementation for is_bottom for consistency

0da6291

outline throws

2dad6c7

loop comment

56c65ce

reduce PR scope, use recursion instead of looping for better inference

652ddec

nsajko added 9 commits February 26, 2025 21:24

fix

54d1a14

fix boostrap

1b734cb

nicer tuple definition, try if this fixes bootstrap

0dc293b

try increasing the limit, hopefully without breaking bootstrap again

1fa91aa

decrease limit, hopefully fixing the bootstrap

71f02db

further decrease limit

7822981

again

f123f50

delete @inlines, maybe that can help without decreasing the limit

d1b6938

sigh

fac1ce7

nsajko force-pushed the prevent_stack_overflow_in_type_promotion branch from 6236d3f to fac1ce7 Compare February 26, 2025 20:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

limit recursion depth to prevent stack overflow in 2-arg `promote_type` #57517

limit recursion depth to prevent stack overflow in 2-arg `promote_type` #57517

nsajko commented Feb 24, 2025 •

edited

Loading

adienes commented Feb 24, 2025

nsajko commented Feb 24, 2025 •

edited

Loading

nsajko commented Feb 25, 2025

adienes commented Feb 25, 2025

nsajko commented Feb 25, 2025

nsajko commented Feb 26, 2025

limit recursion depth to prevent stack overflow in 2-arg promote_type #57517

Are you sure you want to change the base?

limit recursion depth to prevent stack overflow in 2-arg promote_type #57517

Conversation

nsajko commented Feb 24, 2025 • edited Loading

adienes commented Feb 24, 2025

nsajko commented Feb 24, 2025 • edited Loading

nsajko commented Feb 25, 2025

adienes commented Feb 25, 2025

nsajko commented Feb 25, 2025

nsajko commented Feb 26, 2025

limit recursion depth to prevent stack overflow in 2-arg `promote_type` #57517

limit recursion depth to prevent stack overflow in 2-arg `promote_type` #57517

nsajko commented Feb 24, 2025 •

edited

Loading

nsajko commented Feb 24, 2025 •

edited

Loading