Skip to content

Set max_methods to 1 for merge/merge!/mergewith/mergewith!#61330

Merged
oscardssmith merged 1 commit intoJuliaLang:masterfrom
JamesWrigley:merge-max-methods
Mar 25, 2026
Merged

Set max_methods to 1 for merge/merge!/mergewith/mergewith!#61330
oscardssmith merged 1 commit intoJuliaLang:masterfrom
JamesWrigley:merge-max-methods

Conversation

@JamesWrigley
Copy link
Copy Markdown
Member

Should prevent these invalidations seen when loading CurveFit.jl on 1.13:

inserting merge!(m::DataStructures.SortedDict{K, D, Ord}, others::AbstractDict{K, D}...) where {K, D, Ord<:Ordering} @ DataStructures ~/.julia/packages/DataStructures/qUuAY/src/sorted_dict.jl:473 invalidated:
   mt_backedges: 1: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{_A, Base.Compiler.IRShow.var"#30#31"} where _A, ::Any) where {K, V} (0 children)
                 2: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator, ::Any) where {K, V} (0 children)
                 3: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator, ::Any) where {K, V} (0 children)
                 4: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{I, Base.var"#Generator##0#Generator##1"{NonlinearSolveBase.Utils.var"#norm_op##2#norm_op##3"{typeof(+)}}} where I<:(Base.Iterators.Zip{Is} where Is<:Tuple{Base.AbstractBroadcasted, Base.AbstractBroadcasted}), ::Tuple{Tuple{Any, Any}, Tuple{Any, Any}}) where {K, V} (0 children)
                 5: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{_A, F} where {_A, F<:DifferentiationInterface.var"#_prepare_jacobian_aux##0#_prepare_jacobian_aux##1"}, ::Any) where {K, V} (0 children)
                 6: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{_A, F} where {_A, F<:DifferentiationInterface.var"#_prepare_jacobian_aux##12#_prepare_jacobian_aux##13"}, ::Any) where {K, V} (0 children)
                 7: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{I, Base.var"#Generator##0#Generator##1"{NonlinearSolveBase.Utils.var"#norm_op##2#norm_op##3"{typeof(+)}}} where I<:(Base.Iterators.Zip{Is} where Is<:Tuple{Base.AbstractBroadcasted, Vector{Float64}}), ::Tuple{Tuple{Any, Any}, Int64}) where {K, V} (0 children)
   backedges: 1: superseding merge!(d::AbstractDict, others::AbstractDict...) @ Base abstractdict.jl:224 with MethodInstance for merge!(::AbstractDict, ::AbstractDict) (1126 children)

inserting merge(d::OrderedCollections.OrderedDict, others::AbstractDict...) @ OrderedCollections ~/.julia/dev/OrderedCollections/src/ordered_dict.jl:488 invalidated:
   mt_backedges: <38 invalidations I deleted because they have enormous types>
   backedges: 1: superseding merge(d::AbstractDict, others::AbstractDict...) @ Base abstractdict.jl:359 with MethodInstance for merge(::AbstractDict, ::Base.Pairs{Symbol, Union{}, Nothing, @NamedTuple{}}) (21 children)
              2: superseding merge(d::AbstractDict, others::AbstractDict...) @ Base abstractdict.jl:359 with MethodInstance for merge(::AbstractDict, ::Base.Pairs{Symbol, _A, Nothing, A} where {_A, A<:NamedTuple}) (2489 children)

Partial alternative to #61329. I added mergewith() and mergewith!() to be on the safe side.

@JamesWrigley JamesWrigley added the backport 1.13 Change should be backported to release-1.13 label Mar 15, 2026
Copy link
Copy Markdown
Member

@aviatesk aviatesk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original invalidation issue can occur generally for any package that overloads AbstractDict no matter if they implement the overload appropriately, so I believe it needs to be fixed on the Base side.
Setting @max_methods 1 may weaken inference capabilities, but the code affected by this would be type-unstable anyway, so such performance issues should generally be addressed on the package side. There is also a discussion about making @max_methods 1 the global default, which could have a significant impact on the ecosystem. However, I don't see much of a problem with introducing it in a limited fashion for code where actual issues are occurring.

@JamesWrigley
Copy link
Copy Markdown
Member Author

There is also a discussion about making @max_methods 1 the global default, which could have a significant impact on the ecosystem.

That would be amazing, modulo performance hits 😅 I feel like otherwise we'll end up setting max_methods for all interface functions (like a manual version of #59091 (comment)).

@JamesWrigley
Copy link
Copy Markdown
Member Author

I'll merge this in a few days if there's no objections.

@KristofferC KristofferC mentioned this pull request Mar 24, 2026
27 tasks
@oscardssmith oscardssmith merged commit c3b914f into JuliaLang:master Mar 25, 2026
9 checks passed
@JamesWrigley JamesWrigley deleted the merge-max-methods branch March 25, 2026 06:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport 1.13 Change should be backported to release-1.13

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants