From 5bf38ea656f92e42d11407c9a3750c95053393b5 Mon Sep 17 00:00:00 2001 From: Christopher Rowley Date: Sun, 16 Nov 2025 19:00:48 +0000 Subject: [PATCH 01/11] Refactor pyconvert rule scoping --- CHANGELOG.md | 1 + docs/src/conversion-to-julia.md | 25 +-- src/API/exports.jl | 5 - src/API/types.jl | 10 -- src/Convert/Convert.jl | 3 +- src/Convert/ctypes.jl | 22 +-- src/Convert/numpy.jl | 44 ++--- src/Convert/pandas.jl | 6 +- src/Convert/pyconvert.jl | 295 +++++++++++++++++++------------- src/JlWrap/base.jl | 3 +- src/Wrap/Wrap.jl | 51 +++--- test/Convert.jl | 3 +- 12 files changed, 257 insertions(+), 211 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f615e692..622a2be9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ * The vast majority of these changes are breaking, see the [v1 Migration Guide](@ref) for how to upgrade. * Changes to core functionality: * Comparisons like `==(::Py, ::Py)`, `<(::Py, ::Number)`, `isless(::Number, ::Py)` now return `Bool` instead of `Py`. + * `pyconvert` rules are now scoped by target type instead of prioritized; rules are ordered by Python type specificity and creation order. * Changes to `PythonCall.GC` (now more like `Base.GC`): * `enable(true)` replaces `enable()`. * `enable(false)` replaces `disable()`. diff --git a/docs/src/conversion-to-julia.md b/docs/src/conversion-to-julia.md index 5c65a2aa..7d898f27 100644 --- a/docs/src/conversion-to-julia.md +++ b/docs/src/conversion-to-julia.md @@ -2,7 +2,7 @@ ## [Conversion Rules](@id py2jl-conversion) -The following table specifies the conversion rules used whenever converting a Python object to a Julia object. If the initial Python type matches the "From" column and the desired type `T` intersects with the "To" column, then that conversion is attempted. Conversions are tried in priority order, then in specificity order. +The following table specifies the conversion rules used whenever converting a Python object to a Julia object. If the initial Python type matches the "From" column and the desired type `T` intersects with the "To" column, then that conversion is attempted. Rules are ordered by Python type specificity (strict subclassing only) and then by creation order. A rule only applies when the requested target type is a subtype of its scope; unless otherwise noted, the scope matches the type in the "To" column, so those rules apply when you explicitly request that type. From Julia, one can convert Python objects to a desired type using `pyconvert(T, x)` for example. @@ -10,11 +10,11 @@ From Python, the arguments to a Julia function will be converted according to th | From | To | | :----------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------- | -| **Top priority (wrapped values).** | | -| `juliacall.Jl` | `Any` | -| **Very high priority (arrays).** | | +| **Default-scope rules (apply even when converting to `Any`).** | + | +| `juliacall.Jl` | `Any` | | Objects satisfying the buffer or array interface (inc. `bytes`, `bytearray`, `array.array`, `numpy.ndarray`) | `PyArray` | -| **High priority (canonical conversions).** | | +| **Default conversions for common types.** | | `None` | `Nothing` | | `bool` | `Bool` | | `numbers.Integral` (inc. `int`) | `Integer` (prefers `Int`, or `BigInt` on overflow) | @@ -33,8 +33,9 @@ From Python, the arguments to a Julia function will be converted according to th | `numpy.intXX`/`numpy.uintXX`/`numpy.floatXX` | `IntXX`/`UIntXX`/`FloatXX` | | `numpy.datetime64` | `NumpyDates.DateTime64` | | `numpy.timedelta64` | `NumpyDates.TimeDelta64` | -| **Standard priority (other reasonable conversions).** | | -| `None` | `Missing` | +| **Additional conversions requiring matching target scope.** | + | +| `None` | `Missing` (scope `Missing`) | | `bytes` | `Vector{UInt8}`, `Vector{Int8}`, `String` | | `str` | `String`, `Symbol`, `Char`, `Vector{UInt8}`, `Vector{Int8}` | | `range` | `UnitRange` | @@ -52,11 +53,11 @@ From Python, the arguments to a Julia function will be converted according to th | `numpy.bool_`/`numpy.intXX`/`numpy.uintXX`/`numpy.floatXX` | `Bool`, `Integer`, `Rational`, `Real`, `Number` | | `numpy.datetime64` | `NumpyDates.InlineDateTime64`, `Dates.DateTime` | | `numpy.timedelta64` | `NumpyDates.InlineTimeDelta64`, `Dates.Period` | -| Objects satisfying the buffer or array interface | `Array`, `AbstractArray` | -| **Low priority (fallback to `Py`).** | | -| Anything | `Py` | -| **Bottom priority (must be explicitly specified by excluding `Py`).** | | -| Objects satisfying the buffer interface | `PyBuffer` | +| **Fallback conversion.** | + | +| Anything | `Py` | +| **Explicit wrapper conversions (require excluding `Py`).** | + | | Anything | `PyRef` | See [here](@ref python-wrappers) for an explanation of the `Py*` wrapper types (`PyList`, `PyIO`, etc). diff --git a/src/API/exports.jl b/src/API/exports.jl index 0454e0e1..9a529c53 100644 --- a/src/API/exports.jl +++ b/src/API/exports.jl @@ -103,11 +103,6 @@ export pyxor export @pyconvert export pyconvert export pyconvert_add_rule -export PYCONVERT_PRIORITY_ARRAY -export PYCONVERT_PRIORITY_CANONICAL -export PYCONVERT_PRIORITY_FALLBACK -export PYCONVERT_PRIORITY_NORMAL -export PYCONVERT_PRIORITY_WRAP export pyconvert_return export pyconvert_unconverted diff --git a/src/API/types.jl b/src/API/types.jl index 14acece8..79252afb 100644 --- a/src/API/types.jl +++ b/src/API/types.jl @@ -1,13 +1,3 @@ -# Convert - -@enum PyConvertPriority begin - PYCONVERT_PRIORITY_WRAP = 400 - PYCONVERT_PRIORITY_ARRAY = 300 - PYCONVERT_PRIORITY_CANONICAL = 200 - PYCONVERT_PRIORITY_NORMAL = 0 - PYCONVERT_PRIORITY_FALLBACK = -100 -end - # Core """ diff --git a/src/Convert/Convert.jl b/src/Convert/Convert.jl index fc506fc6..c3e7e0cd 100644 --- a/src/Convert/Convert.jl +++ b/src/Convert/Convert.jl @@ -18,8 +18,7 @@ import ..PythonCall: pyconvert_add_rule, pyconvert_return, pyconvert_unconverted, - pyconvert, - PyConvertPriority + pyconvert export pyconvert_isunconverted, pyconvert_result, diff --git a/src/Convert/ctypes.jl b/src/Convert/ctypes.jl index d18bbc74..e0a078c1 100644 --- a/src/Convert/ctypes.jl +++ b/src/Convert/ctypes.jl @@ -47,18 +47,18 @@ function init_ctypes() rule = pyconvert_rule_ctypessimplevalue{T,false}() saferule = pyconvert_rule_ctypessimplevalue{T,true}() - t == "char_p" && pyconvert_add_rule(name, Cstring, saferule) - t == "wchar_p" && pyconvert_add_rule(name, Cwstring, saferule) - pyconvert_add_rule(name, T, saferule) - isuint && pyconvert_add_rule(name, UInt, sizeof(T) ≤ sizeof(UInt) ? saferule : rule) - isuint && pyconvert_add_rule(name, Int, sizeof(T) < sizeof(Int) ? saferule : rule) + t == "char_p" && pyconvert_add_rule(saferule, name, Cstring) + t == "wchar_p" && pyconvert_add_rule(saferule, name, Cwstring) + pyconvert_add_rule(saferule, name, T) + isuint && pyconvert_add_rule(sizeof(T) ≤ sizeof(UInt) ? saferule : rule, name, UInt) + isuint && pyconvert_add_rule(sizeof(T) < sizeof(Int) ? saferule : rule, name, Int) isint && !isuint && - pyconvert_add_rule(name, Int, sizeof(T) ≤ sizeof(Int) ? saferule : rule) - isint && pyconvert_add_rule(name, Integer, rule) - isfloat && pyconvert_add_rule(name, Float64, saferule) - isreal && pyconvert_add_rule(name, Real, rule) - isnumber && pyconvert_add_rule(name, Number, rule) - isptr && pyconvert_add_rule(name, Ptr, saferule) + pyconvert_add_rule(sizeof(T) ≤ sizeof(Int) ? saferule : rule, name, Int) + isint && pyconvert_add_rule(rule, name, Integer) + isfloat && pyconvert_add_rule(saferule, name, Float64) + isreal && pyconvert_add_rule(rule, name, Real) + isnumber && pyconvert_add_rule(rule, name, Number) + isptr && pyconvert_add_rule(saferule, name, Ptr) end end diff --git a/src/Convert/numpy.jl b/src/Convert/numpy.jl index 73383d54..c73b0dce 100644 --- a/src/Convert/numpy.jl +++ b/src/Convert/numpy.jl @@ -112,49 +112,49 @@ function init_numpy() rule = pyconvert_rule_numpysimplevalue{T,false}() saferule = pyconvert_rule_numpysimplevalue{T,true}() - pyconvert_add_rule(name, T, saferule, PYCONVERT_PRIORITY_ARRAY) - isuint && pyconvert_add_rule(name, UInt, sizeof(T) ≤ sizeof(UInt) ? saferule : rule) - isuint && pyconvert_add_rule(name, Int, sizeof(T) < sizeof(Int) ? saferule : rule) + pyconvert_add_rule(saferule, name, T, Any) + isuint && pyconvert_add_rule(sizeof(T) ≤ sizeof(UInt) ? saferule : rule, name, UInt) + isuint && pyconvert_add_rule(sizeof(T) < sizeof(Int) ? saferule : rule, name, Int) isint && !isuint && - pyconvert_add_rule(name, Int, sizeof(T) ≤ sizeof(Int) ? saferule : rule) - isint && pyconvert_add_rule(name, Integer, rule) - isfloat && pyconvert_add_rule(name, Float64, saferule) - isreal && pyconvert_add_rule(name, Real, rule) - iscomplex && pyconvert_add_rule(name, ComplexF64, saferule) - iscomplex && pyconvert_add_rule(name, Complex, rule) - isnumber && pyconvert_add_rule(name, Number, rule) + pyconvert_add_rule(sizeof(T) ≤ sizeof(Int) ? saferule : rule, name, Int) + isint && pyconvert_add_rule(rule, name, Integer) + isfloat && pyconvert_add_rule(saferule, name, Float64) + isreal && pyconvert_add_rule(rule, name, Real) + iscomplex && pyconvert_add_rule(saferule, name, ComplexF64) + iscomplex && pyconvert_add_rule(rule, name, Complex) + isnumber && pyconvert_add_rule(rule, name, Number) end # datetime64 pyconvert_add_rule( + pyconvert_rule_datetime64, "numpy:datetime64", DateTime64, - pyconvert_rule_datetime64, - PYCONVERT_PRIORITY_ARRAY, + Any, ) - pyconvert_add_rule("numpy:datetime64", InlineDateTime64, pyconvert_rule_datetime64) + pyconvert_add_rule(pyconvert_rule_datetime64, "numpy:datetime64", InlineDateTime64) pyconvert_add_rule( + pyconvert_rule_datetime64, "numpy:datetime64", NumpyDates.DatesInstant, - pyconvert_rule_datetime64, ) - pyconvert_add_rule("numpy:datetime64", Missing, pyconvert_rule_datetime64) - pyconvert_add_rule("numpy:datetime64", Nothing, pyconvert_rule_datetime64) + pyconvert_add_rule(pyconvert_rule_datetime64, "numpy:datetime64", Missing, Missing) + pyconvert_add_rule(pyconvert_rule_datetime64, "numpy:datetime64", Nothing, Nothing) # timedelta64 pyconvert_add_rule( + pyconvert_rule_timedelta64, "numpy:timedelta64", TimeDelta64, - pyconvert_rule_timedelta64, - PYCONVERT_PRIORITY_ARRAY, + Any, ) - pyconvert_add_rule("numpy:timedelta64", InlineTimeDelta64, pyconvert_rule_timedelta64) + pyconvert_add_rule(pyconvert_rule_timedelta64, "numpy:timedelta64", InlineTimeDelta64) pyconvert_add_rule( + pyconvert_rule_timedelta64, "numpy:timedelta64", NumpyDates.DatesPeriod, - pyconvert_rule_timedelta64, ) - pyconvert_add_rule("numpy:timedelta64", Missing, pyconvert_rule_timedelta64) - pyconvert_add_rule("numpy:timedelta64", Nothing, pyconvert_rule_timedelta64) + pyconvert_add_rule(pyconvert_rule_timedelta64, "numpy:timedelta64", Missing, Missing) + pyconvert_add_rule(pyconvert_rule_timedelta64, "numpy:timedelta64", Nothing, Nothing) end diff --git a/src/Convert/pandas.jl b/src/Convert/pandas.jl index 65786f1c..1604979e 100644 --- a/src/Convert/pandas.jl +++ b/src/Convert/pandas.jl @@ -3,10 +3,10 @@ pyconvert_rule_pandas_na(::Type{Missing}, x::Py) = pyconvert_return(missing) function init_pandas() pyconvert_add_rule( + pyconvert_rule_pandas_na, "pandas._libs.missing:NAType", Missing, - pyconvert_rule_pandas_na, - PYCONVERT_PRIORITY_CANONICAL, + Any, ) - pyconvert_add_rule("pandas._libs.missing:NAType", Nothing, pyconvert_rule_pandas_na) + pyconvert_add_rule(pyconvert_rule_pandas_na, "pandas._libs.missing:NAType", Nothing, Nothing) end diff --git a/src/Convert/pyconvert.jl b/src/Convert/pyconvert.jl index 2b03eb72..c7bc97eb 100644 --- a/src/Convert/pyconvert.jl +++ b/src/Convert/pyconvert.jl @@ -1,14 +1,17 @@ struct PyConvertRule + tname::String type::Type + scope::Type func::Function - priority::PyConvertPriority + order::Int end const PYCONVERT_RULES = Dict{String,Vector{PyConvertRule}}() +const PYCONVERT_RULE_ORDER = Ref{Int}(0) const PYCONVERT_EXTRATYPES = Py[] """ - pyconvert_add_rule(tname::String, T::Type, func::Function, priority::PyConvertPriority=PYCONVERT_PRIORITY_NORMAL) + pyconvert_add_rule(func::Function, tname::String, ::Type{T}, ::Type{S}=T) where {T,S} Add a new conversion rule for `pyconvert`. @@ -19,14 +22,15 @@ Add a new conversion rule for `pyconvert`. Python objects of this type. - `T` is a Julia type, such that this rule only applies when the target type intersects with `T`. +- `S` is a Julia type, such that this rule only applies when the target type is a subtype + of `S` (or a union whose components include a subtype of `S`). - `func` is the function implementing the rule. -- `priority` determines whether to prioritise this rule above others. When `pyconvert(R, x)` is called, all rules such that `typeintersect(T, R) != Union{}` -and `pyisinstance(x, t)` are considered. These rules are sorted first by priority, -then by the specificity of `t` (e.g. `bool` is more specific than `int` is more specific -than `object`) then by the order they were added. The rules are tried in turn until one -succeeds. +and `pyisinstance(x, t)` are considered. It also requires that `R <: S`, or if `R` is a +union then at least one component satisfies this property. These rules are sorted first +by the specificity of `t` (strict subclassing only) then by the order they were added. +The rules are tried in turn until one succeeds. ### Implementing `func` @@ -40,32 +44,17 @@ It must return one of: The target type `S` is never a union or the empty type, i.e. it is always a data type or union-all. -### Priority - -Most rules should have priority `PYCONVERT_PRIORITY_NORMAL` (the default) which is for any -reasonable conversion rule. - -Use priority `PYCONVERT_PRIORITY_CANONICAL` for **canonical** conversion rules. Immutable -objects may be canonically converted to their corresponding Julia type, such as `int` to -`Integer`. Mutable objects **must** be converted to a wrapper type, such that the original -Python object can be retrieved. For example a `list` is canonically converted to `PyList` -and not to a `Vector`. There should not be more than one canonical conversion rule for a -given Python type. - -Other priorities are reserved for internal use. """ -function pyconvert_add_rule( - pytypename::String, - type::Type, - func::Function, - priority::PyConvertPriority = PYCONVERT_PRIORITY_NORMAL, -) - @nospecialize type func +function pyconvert_add_rule(func::Function, pytypename::String, type::Type, scope::Type = type) + @nospecialize type func scope + type <: scope || error("pyconvert rule must satisfy T <: S") + order = (PYCONVERT_RULE_ORDER[] += 1) push!( get!(Vector{PyConvertRule}, PYCONVERT_RULES, pytypename), - PyConvertRule(type, func, priority), + PyConvertRule(pytypename, type, scope, func, order), ) empty!.(values(PYCONVERT_RULES_CACHE)) + empty!(PYCONVERT_PREFERRED_TYPE) return end @@ -148,6 +137,9 @@ function pyconvert_typename(t::Py) return "$m:$n" end +pyconvert_is_special_tname(tname::String) = + tname in ("", "", "", "") + function _pyconvert_get_rules(pytype::Py) pyisin(x, ys) = any(pyis(x, y) for y in ys) @@ -237,20 +229,36 @@ function _pyconvert_get_rules(pytype::Py) end # flatten to get the MRO as a list of strings - mro = String[x for xs in xmro for x in xs] + mro_strings = String[x for xs in xmro for x in xs] + + typemap = Dict{String,Py}() + for t in mro + typemap[pyconvert_typename(t)] = t + end # get corresponding rules - rules = PyConvertRule[ - rule for tname in mro for - rule in get!(Vector{PyConvertRule}, PYCONVERT_RULES, tname) + rules = PyConvertRuleInfo[ + PyConvertRuleInfo( + tname, + get(typemap, tname, PyNULL), + rule.type, + rule.scope, + rule.func, + rule.order, + ) for tname in mro_strings for rule in get!(Vector{PyConvertRule}, PYCONVERT_RULES, tname) ] - # order the rules by priority, then by original order - order = sort(axes(rules, 1), by = i -> (rules[i].priority, -i), rev = true) - rules = rules[order] + @debug "pyconvert" pytype mro = join(mro_strings, " ") + return rules, typemap +end - @debug "pyconvert" pytype mro = join(mro, " ") - return rules +struct PyConvertRuleInfo + tname::String + pytype::Py + type::Type + scope::Type + func::Function + order::Int end const PYCONVERT_PREFERRED_TYPE = Dict{Py,Type}() @@ -260,43 +268,26 @@ pyconvert_preferred_type(pytype::Py) = if pyissubclass(pytype, pybuiltins.int) Union{Int,BigInt} else - _pyconvert_get_rules(pytype)[1].type + pyconvert_get_rules_info(Any, pytype)[1].type end end function pyconvert_get_rules(type::Type, pytype::Py) @nospecialize type - # this could be cached - rules = _pyconvert_get_rules(pytype) - - # intersect rules with type - rules = PyConvertRule[ - PyConvertRule(typeintersect(rule.type, type), rule.func, rule.priority) for - rule in rules - ] - - # explode out unions - rules = [ - PyConvertRule(type, rule.func, rule.priority) for rule in rules for - type in Utils.explode_union(rule.type) - ] - - # filter out empty rules - rules = [rule for rule in rules if rule.type != Union{}] - - # filter out repeated rules - rules = [ - rule for (i, rule) in enumerate(rules) if !any( - (rule.func === rules[j].func) && ((rule.type) <: (rules[j].type)) for - j = 1:(i-1) - ) - ] + rules = pyconvert_get_rules_info(type, pytype) @debug "pyconvert" type rules return Function[pyconvert_fix(rule.type, rule.func) for rule in rules] end +function pyconvert_get_rules_info(type::Type, pytype::Py) + @nospecialize type + rules, typemap = _pyconvert_get_rules(pytype) + rules = _pyconvert_filter_rules(type, rules) + return _pyconvert_order_rules(rules, typemap) +end + pyconvert_fix(::Type{T}, func) where {T} = x -> func(T, x) const PYCONVERT_RULES_CACHE = Dict{Type,Dict{C.PyPtr,Vector{Function}}}() @@ -304,6 +295,85 @@ const PYCONVERT_RULES_CACHE = Dict{Type,Dict{C.PyPtr,Vector{Function}}}() @generated pyconvert_rules_cache(::Type{T}) where {T} = get!(Dict{C.PyPtr,Vector{Function}}, PYCONVERT_RULES_CACHE, T) +function _pyconvert_type_in_scope(::Type{R}, ::Type{S}) where {R,S} + if R isa Union + any(_pyconvert_type_in_scope(T, S) for T in Utils.explode_union(R)) + else + R <: S + end +end + +function _pyconvert_filter_rules(type::Type, rules::Vector{PyConvertRuleInfo}) + @nospecialize type + filtered = PyConvertRuleInfo[] + for rule in rules + T = typeintersect(rule.type, type) + T == Union{} && continue + _pyconvert_type_in_scope(type, rule.scope) || continue + for U in Utils.explode_union(T) + U == Union{} && continue + push!(filtered, PyConvertRuleInfo(rule.tname, rule.pytype, U, rule.scope, rule.func, rule.order)) + end + end + + filtered = [ + rule for (i, rule) in enumerate(filtered) if !any( + (rule.func === filtered[j].func) && (rule.type <: filtered[j].type) for j = 1:(i - 1) + ) + ] + return filtered +end + +function _pyconvert_tname_subclass(t1::String, t2::String, typemap::Dict{String,Py}) + t1 == t2 && return false + py1 = get(typemap, t1, PyNULL) + py2 = get(typemap, t2, PyNULL) + + if t2 == "" + return (t1 != "") && !pyisnull(py1) && (C.PyType_CheckBuffer(py1) != 0) + elseif t2 == "" + return (t1 != "") && !pyisnull(py1) && pyhasattr(py1, "__array__") + elseif t2 == "" + return (t1 != "") && !pyisnull(py1) && pyhasattr(py1, "__array_interface__") + elseif t2 == "" + return (t1 != "") && !pyisnull(py1) && pyhasattr(py1, "__array_struct__") + elseif pyisnull(py1) || pyisnull(py2) + return false + else + return pyissubclass(py1, py2) + end +end + +function _pyconvert_order_rules(rules::Vector{PyConvertRuleInfo}, typemap::Dict{String,Py}) + incoming = Dict{Int,Vector{Int}}() + for i in eachindex(rules) + incoming[i] = Int[] + end + for (i, ri) in pairs(rules), (j, rj) in pairs(rules) + (i == j) && continue + if _pyconvert_tname_subclass(ri.tname, rj.tname, typemap) + push!(incoming[j], i) + end + end + + ordered = PyConvertRuleInfo[] + remaining = collect(keys(incoming)) + while !isempty(remaining) + available = [i for i in remaining if isempty(incoming[i])] + isempty(available) && error("pyconvert rule ordering cycle detected") + sort!(available, by = i -> rules[i].order) + append!(ordered, rules[available]) + for i in available + delete!(incoming, i) + end + filter!(i -> haskey(incoming, i), remaining) + for deps in values(incoming) + filter!(i -> i ∉ available, deps) + end + end + return ordered +end + function pyconvert_rule_fast(::Type{T}, x::Py) where {T} if T isa Union a = pyconvert_rule_fast(T.a, x)::pyconvert_returntype(T.a) @@ -418,99 +488,92 @@ function init_pyconvert() pyimport("collections.abc" => ("Iterable", "Sequence", "Set", "Mapping"))..., ) - priority = PYCONVERT_PRIORITY_CANONICAL - pyconvert_add_rule("builtins:NoneType", Nothing, pyconvert_rule_none, priority) - pyconvert_add_rule("builtins:bool", Bool, pyconvert_rule_bool, priority) - pyconvert_add_rule("builtins:float", Float64, pyconvert_rule_float, priority) + pyconvert_add_rule(pyconvert_rule_none, "builtins:NoneType", Nothing, Any) + pyconvert_add_rule(pyconvert_rule_bool, "builtins:bool", Bool, Any) + pyconvert_add_rule(pyconvert_rule_float, "builtins:float", Float64, Any) pyconvert_add_rule( + pyconvert_rule_complex, "builtins:complex", Complex{Float64}, - pyconvert_rule_complex, - priority, + Any, ) - pyconvert_add_rule("numbers:Integral", Integer, pyconvert_rule_int, priority) - pyconvert_add_rule("builtins:str", String, pyconvert_rule_str, priority) + pyconvert_add_rule(pyconvert_rule_int, "numbers:Integral", Integer, Any) + pyconvert_add_rule(pyconvert_rule_str, "builtins:str", String, Any) pyconvert_add_rule( + pyconvert_rule_bytes, "builtins:bytes", Base.CodeUnits{UInt8,String}, - pyconvert_rule_bytes, - priority, + Any, ) pyconvert_add_rule( + pyconvert_rule_range, "builtins:range", StepRange{<:Integer,<:Integer}, - pyconvert_rule_range, - priority, + Any, ) pyconvert_add_rule( + pyconvert_rule_fraction, "numbers:Rational", Rational{<:Integer}, - pyconvert_rule_fraction, - priority, + Any, ) - pyconvert_add_rule("builtins:tuple", NamedTuple, pyconvert_rule_iterable, priority) - pyconvert_add_rule("builtins:tuple", Tuple, pyconvert_rule_iterable, priority) - pyconvert_add_rule("datetime:datetime", DateTime, pyconvert_rule_datetime, priority) - pyconvert_add_rule("datetime:date", Date, pyconvert_rule_date, priority) - pyconvert_add_rule("datetime:time", Time, pyconvert_rule_time, priority) + pyconvert_add_rule(pyconvert_rule_iterable, "builtins:tuple", NamedTuple, Any) + pyconvert_add_rule(pyconvert_rule_iterable, "builtins:tuple", Tuple, Any) + pyconvert_add_rule(pyconvert_rule_datetime, "datetime:datetime", DateTime, Any) + pyconvert_add_rule(pyconvert_rule_date, "datetime:date", Date, Any) + pyconvert_add_rule(pyconvert_rule_time, "datetime:time", Time, Any) pyconvert_add_rule( + pyconvert_rule_timedelta, "datetime:timedelta", Microsecond, - pyconvert_rule_timedelta, - priority, + Any, ) pyconvert_add_rule( + pyconvert_rule_exception, "builtins:BaseException", PyException, - pyconvert_rule_exception, - priority, + Any, ) - priority = PYCONVERT_PRIORITY_NORMAL - pyconvert_add_rule("builtins:NoneType", Missing, pyconvert_rule_none, priority) - pyconvert_add_rule("builtins:bool", Number, pyconvert_rule_bool, priority) - pyconvert_add_rule("numbers:Real", Number, pyconvert_rule_float, priority) - pyconvert_add_rule("builtins:float", Nothing, pyconvert_rule_float, priority) - pyconvert_add_rule("builtins:float", Missing, pyconvert_rule_float, priority) - pyconvert_add_rule("numbers:Complex", Number, pyconvert_rule_complex, priority) - pyconvert_add_rule("numbers:Integral", Number, pyconvert_rule_int, priority) - pyconvert_add_rule("builtins:str", Symbol, pyconvert_rule_str, priority) - pyconvert_add_rule("builtins:str", Char, pyconvert_rule_str, priority) - pyconvert_add_rule("builtins:bytes", Vector{UInt8}, pyconvert_rule_bytes, priority) + pyconvert_add_rule(pyconvert_rule_none, "builtins:NoneType", Missing, Missing) + pyconvert_add_rule(pyconvert_rule_bool, "builtins:bool", Number, Number) + pyconvert_add_rule(pyconvert_rule_float, "numbers:Real", Number, Number) + pyconvert_add_rule(pyconvert_rule_float, "builtins:float", Nothing, Nothing) + pyconvert_add_rule(pyconvert_rule_float, "builtins:float", Missing, Missing) + pyconvert_add_rule(pyconvert_rule_complex, "numbers:Complex", Number, Number) + pyconvert_add_rule(pyconvert_rule_int, "numbers:Integral", Number, Number) + pyconvert_add_rule(pyconvert_rule_str, "builtins:str", Symbol) + pyconvert_add_rule(pyconvert_rule_str, "builtins:str", Char) + pyconvert_add_rule(pyconvert_rule_bytes, "builtins:bytes", Vector{UInt8}) pyconvert_add_rule( + pyconvert_rule_range, "builtins:range", UnitRange{<:Integer}, - pyconvert_rule_range, - priority, ) - pyconvert_add_rule("numbers:Rational", Number, pyconvert_rule_fraction, priority) + pyconvert_add_rule(pyconvert_rule_fraction, "numbers:Rational", Number, Number) pyconvert_add_rule( + pyconvert_rule_iterable, "collections.abc:Iterable", Vector, - pyconvert_rule_iterable, - priority, ) - pyconvert_add_rule("collections.abc:Iterable", Tuple, pyconvert_rule_iterable, priority) - pyconvert_add_rule("collections.abc:Iterable", Pair, pyconvert_rule_iterable, priority) - pyconvert_add_rule("collections.abc:Iterable", Set, pyconvert_rule_iterable, priority) + pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", Tuple) + pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", Pair) + pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", Set) pyconvert_add_rule( + pyconvert_rule_iterable, "collections.abc:Sequence", Vector, - pyconvert_rule_iterable, - priority, ) - pyconvert_add_rule("collections.abc:Sequence", Tuple, pyconvert_rule_iterable, priority) - pyconvert_add_rule("collections.abc:Set", Set, pyconvert_rule_iterable, priority) - pyconvert_add_rule("collections.abc:Mapping", Dict, pyconvert_rule_mapping, priority) + pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Sequence", Tuple) + pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Set", Set) + pyconvert_add_rule(pyconvert_rule_mapping, "collections.abc:Mapping", Dict) pyconvert_add_rule( + pyconvert_rule_timedelta, "datetime:timedelta", Millisecond, - pyconvert_rule_timedelta, - priority, ) - pyconvert_add_rule("datetime:timedelta", Second, pyconvert_rule_timedelta, priority) - pyconvert_add_rule("datetime:timedelta", Nanosecond, pyconvert_rule_timedelta, priority) + pyconvert_add_rule(pyconvert_rule_timedelta, "datetime:timedelta", Second) + pyconvert_add_rule(pyconvert_rule_timedelta, "datetime:timedelta", Nanosecond) - priority = PYCONVERT_PRIORITY_FALLBACK - pyconvert_add_rule("builtins:object", Py, pyconvert_rule_object, priority) + pyconvert_add_rule(pyconvert_rule_object, "builtins:object", Py, Any) end diff --git a/src/JlWrap/base.jl b/src/JlWrap/base.jl index 2497f4b0..cfd9e1bb 100644 --- a/src/JlWrap/base.jl +++ b/src/JlWrap/base.jl @@ -25,8 +25,7 @@ function init_base() pyjuliacallmodule.JlBase = pyjlbasetype # conversion rule - priority = PYCONVERT_PRIORITY_WRAP - pyconvert_add_rule("juliacall:JlBase", Any, pyconvert_rule_jlvalue, priority) + pyconvert_add_rule(pyconvert_rule_jlvalue, "juliacall:JlBase", Any, Any) end pyconvert_rule_jlvalue(::Type{T}, x::Py) where {T} = diff --git a/src/Wrap/Wrap.jl b/src/Wrap/Wrap.jl index e67129cb..4be26365 100644 --- a/src/Wrap/Wrap.jl +++ b/src/Wrap/Wrap.jl @@ -32,51 +32,48 @@ include("PyTable.jl") include("PyPandasDataFrame.jl") function __init__() - priority = PYCONVERT_PRIORITY_ARRAY - pyconvert_add_rule("", PyArray, pyconvert_rule_array_nocopy, priority) - pyconvert_add_rule("", PyArray, pyconvert_rule_array_nocopy, priority) - pyconvert_add_rule("", PyArray, pyconvert_rule_array_nocopy, priority) - pyconvert_add_rule("", PyArray, pyconvert_rule_array_nocopy, priority) + pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) + pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) + pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) + pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) - priority = PYCONVERT_PRIORITY_CANONICAL pyconvert_add_rule( + pyconvert_rule_iterable, "collections.abc:Iterable", PyIterable, - pyconvert_rule_iterable, - priority, + Any, ) pyconvert_add_rule( + pyconvert_rule_sequence, "collections.abc:Sequence", PyList, - pyconvert_rule_sequence, - priority, + Any, ) - pyconvert_add_rule("collections.abc:Set", PySet, pyconvert_rule_set, priority) - pyconvert_add_rule("collections.abc:Mapping", PyDict, pyconvert_rule_mapping, priority) - pyconvert_add_rule("io:IOBase", PyIO, pyconvert_rule_io, priority) - pyconvert_add_rule("_io:_IOBase", PyIO, pyconvert_rule_io, priority) + pyconvert_add_rule(pyconvert_rule_set, "collections.abc:Set", PySet, Any) + pyconvert_add_rule(pyconvert_rule_mapping, "collections.abc:Mapping", PyDict, Any) + pyconvert_add_rule(pyconvert_rule_io, "io:IOBase", PyIO, Any) + pyconvert_add_rule(pyconvert_rule_io, "_io:_IOBase", PyIO, Any) pyconvert_add_rule( + pyconvert_rule_pandasdataframe, "pandas.core.frame:DataFrame", PyPandasDataFrame, - pyconvert_rule_pandasdataframe, - priority, + Any, ) pyconvert_add_rule( + pyconvert_rule_sequence, "pandas.core.arrays.base:ExtensionArray", PyList, - pyconvert_rule_sequence, - priority, + Any, ) - priority = PYCONVERT_PRIORITY_NORMAL - pyconvert_add_rule("", Array, pyconvert_rule_array, priority) - pyconvert_add_rule("", Array, pyconvert_rule_array, priority) - pyconvert_add_rule("", Array, pyconvert_rule_array, priority) - pyconvert_add_rule("", Array, pyconvert_rule_array, priority) - pyconvert_add_rule("", AbstractArray, pyconvert_rule_array, priority) - pyconvert_add_rule("", AbstractArray, pyconvert_rule_array, priority) - pyconvert_add_rule("", AbstractArray, pyconvert_rule_array, priority) - pyconvert_add_rule("", AbstractArray, pyconvert_rule_array, priority) + pyconvert_add_rule(pyconvert_rule_array, "", Array) + pyconvert_add_rule(pyconvert_rule_array, "", Array) + pyconvert_add_rule(pyconvert_rule_array, "", Array) + pyconvert_add_rule(pyconvert_rule_array, "", Array) + pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray) + pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray) + pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray) + pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray) end end diff --git a/test/Convert.jl b/test/Convert.jl index 4f137459..45396ee0 100644 --- a/test/Convert.jl +++ b/test/Convert.jl @@ -318,9 +318,10 @@ end @test pyconvert(Any, x) === x # This test has a side effect of influencing the rules cache t = pytype(x) PythonCall.pyconvert_add_rule( + (_, _) -> "Hello!!", "$(t.__module__):$(t.__qualname__)", String, - (_, _) -> "Hello!!", + Any, ) @test pyconvert(String, x) == "Hello!!" @test pyconvert(Any, x) == "Hello!!" # Broken before PR #365 From 9a2ec5aea591421bc0e9cb32df9146d5faa6a6b4 Mon Sep 17 00:00:00 2001 From: Christopher Rowley Date: Sun, 16 Nov 2025 20:03:25 +0000 Subject: [PATCH 02/11] Adjust wrapper rule scopes and CI config --- .github/workflows/tests-nightly.yml | 1 - .github/workflows/tests.yml | 1 - src/Wrap/Wrap.jl | 24 ++++++------------------ 3 files changed, 6 insertions(+), 20 deletions(-) diff --git a/.github/workflows/tests-nightly.yml b/.github/workflows/tests-nightly.yml index 3f2c7c0b..d4ce1914 100644 --- a/.github/workflows/tests-nightly.yml +++ b/.github/workflows/tests-nightly.yml @@ -37,7 +37,6 @@ jobs: - uses: julia-actions/julia-buildpkg@v1 - uses: julia-actions/julia-runtest@v1 env: - JULIA_DEBUG: PythonCall JULIA_NUM_THREADS: '2' - uses: julia-actions/julia-processcoverage@v1 - uses: codecov/codecov-action@v5 diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 10194356..aa78d53d 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -54,7 +54,6 @@ jobs: - name: Run tests uses: julia-actions/julia-runtest@v1 env: - JULIA_DEBUG: PythonCall JULIA_NUM_THREADS: '2' PYTHON: python JULIA_PYTHONCALL_EXE: ${{ matrix.pythonexe }} diff --git a/src/Wrap/Wrap.jl b/src/Wrap/Wrap.jl index 4be26365..182f547c 100644 --- a/src/Wrap/Wrap.jl +++ b/src/Wrap/Wrap.jl @@ -37,33 +37,21 @@ function __init__() pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) - pyconvert_add_rule( - pyconvert_rule_iterable, - "collections.abc:Iterable", - PyIterable, - Any, - ) - pyconvert_add_rule( - pyconvert_rule_sequence, - "collections.abc:Sequence", - PyList, - Any, - ) - pyconvert_add_rule(pyconvert_rule_set, "collections.abc:Set", PySet, Any) - pyconvert_add_rule(pyconvert_rule_mapping, "collections.abc:Mapping", PyDict, Any) - pyconvert_add_rule(pyconvert_rule_io, "io:IOBase", PyIO, Any) - pyconvert_add_rule(pyconvert_rule_io, "_io:_IOBase", PyIO, Any) + pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", PyIterable) + pyconvert_add_rule(pyconvert_rule_sequence, "collections.abc:Sequence", PyList) + pyconvert_add_rule(pyconvert_rule_set, "collections.abc:Set", PySet) + pyconvert_add_rule(pyconvert_rule_mapping, "collections.abc:Mapping", PyDict) + pyconvert_add_rule(pyconvert_rule_io, "io:IOBase", PyIO) + pyconvert_add_rule(pyconvert_rule_io, "_io:_IOBase", PyIO) pyconvert_add_rule( pyconvert_rule_pandasdataframe, "pandas.core.frame:DataFrame", PyPandasDataFrame, - Any, ) pyconvert_add_rule( pyconvert_rule_sequence, "pandas.core.arrays.base:ExtensionArray", PyList, - Any, ) pyconvert_add_rule(pyconvert_rule_array, "", Array) From 37a4f5a8557a0a72402ecaa8935a119c54c4f195 Mon Sep 17 00:00:00 2001 From: Christopher Rowley Date: Sun, 16 Nov 2025 21:17:03 +0000 Subject: [PATCH 03/11] Add guarded conversion tests --- test/Convert.jl | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/test/Convert.jl b/test/Convert.jl index 45396ee0..cc5a71bc 100644 --- a/test/Convert.jl +++ b/test/Convert.jl @@ -265,6 +265,48 @@ end @test_throws Exception pyconvert(Microsecond, td(days = -200_000_000)) end +@testitem "ctypes scalars" begin + ctypes = pyimport("ctypes") + + @test pyconvert(Int, ctypes.c_int(42)) === 42 + @test pyconvert(Float64, ctypes.c_double(3.25)) === 3.25 + @test pyconvert(Ptr{Cvoid}, ctypes.c_void_p(0)) == C_NULL +end + +@testitem "numpy scalars" setup=[Setup] begin + if Setup.devdeps + np = try + pyimport("numpy") + catch + @test_skip "numpy not available" + return + end + + @test pyconvert(Int, np.int64(5)) === 5 + @test pyconvert(Float32, np.float32(1.25)) === Float32(1.25) + @test pyconvert(Missing, np.datetime64("NaT")) === missing + @test pyconvert(Missing, np.timedelta64("NaT")) === missing + else + @test_skip Setup.devdeps + end +end + +@testitem "pandas NA" setup=[Setup] begin + if Setup.devdeps + pd = try + pyimport("pandas") + catch + @test_skip "pandas not available" + return + end + + @test pyconvert(Missing, pd.NA) === missing + @test pyconvert(Nothing, pd.NA) === nothing + else + @test_skip Setup.devdeps + end +end + @testitem "timedelta → Millisecond" begin using Dates td = pyimport("datetime").timedelta From 4beb74be7336de17ace84d44060a8f4191528826 Mon Sep 17 00:00:00 2001 From: Christopher Rowley Date: Sun, 16 Nov 2025 21:33:28 +0000 Subject: [PATCH 04/11] Ensure devdeps include pandas --- CondaPkg.toml | 1 + test/Convert.jl | 14 ++------------ 2 files changed, 3 insertions(+), 12 deletions(-) diff --git a/CondaPkg.toml b/CondaPkg.toml index e5416bc9..34133a30 100644 --- a/CondaPkg.toml +++ b/CondaPkg.toml @@ -15,5 +15,6 @@ version = ">=3.10,<3.14" [dev.deps] matplotlib = "" numpy = "" +pandas = "" pyside6 = "" python = "<3.14" diff --git a/test/Convert.jl b/test/Convert.jl index cc5a71bc..d3404660 100644 --- a/test/Convert.jl +++ b/test/Convert.jl @@ -275,12 +275,7 @@ end @testitem "numpy scalars" setup=[Setup] begin if Setup.devdeps - np = try - pyimport("numpy") - catch - @test_skip "numpy not available" - return - end + np = pyimport("numpy") @test pyconvert(Int, np.int64(5)) === 5 @test pyconvert(Float32, np.float32(1.25)) === Float32(1.25) @@ -293,12 +288,7 @@ end @testitem "pandas NA" setup=[Setup] begin if Setup.devdeps - pd = try - pyimport("pandas") - catch - @test_skip "pandas not available" - return - end + pd = pyimport("pandas") @test pyconvert(Missing, pd.NA) === missing @test pyconvert(Nothing, pd.NA) === nothing From 9e6a96886bb59a53d82b8c7fc46a4a2d425b7ffc Mon Sep 17 00:00:00 2001 From: Christopher Rowley Date: Sun, 16 Nov 2025 21:42:32 +0000 Subject: [PATCH 05/11] Simplify pyconvert rule type collection --- src/Convert/pyconvert.jl | 127 +++++++++++---------------------------- 1 file changed, 36 insertions(+), 91 deletions(-) diff --git a/src/Convert/pyconvert.jl b/src/Convert/pyconvert.jl index c7bc97eb..9ccfcb71 100644 --- a/src/Convert/pyconvert.jl +++ b/src/Convert/pyconvert.jl @@ -140,103 +140,49 @@ end pyconvert_is_special_tname(tname::String) = tname in ("", "", "", "") -function _pyconvert_get_rules(pytype::Py) - pyisin(x, ys) = any(pyis(x, y) for y in ys) - - # get the MROs of all base types we are considering - omro = collect(pytype.__mro__) - basetypes = Py[pytype] - basemros = Vector{Py}[omro] - for xtype in PYCONVERT_EXTRATYPES - # find the topmost supertype of - xbase = PyNULL - for base in omro - if pyissubclass(base, xtype) - xbase = base - end +function _pyconvert_collect_supertypes(pytype::Py) + seen = Set{C.PyPtr}() + queue = Py[pytype] + types = Py[] + while !isempty(queue) + t = pop!(queue) + ptr = C.PyPtr(t) + ptr ∈ seen && continue + push!(seen, ptr) + push!(types, t) + if pyhasattr(t, "__bases__") + append!(queue, (Py(b) for b in t.__bases__)) end - if !pyisnull(xbase) - push!(basetypes, xtype) - xmro = collect(xtype.__mro__) - pyisin(xbase, xmro) || pushfirst!(xmro, xbase) - push!(basemros, xmro) - end - end - for xbase in basetypes[2:end] - push!(basemros, [xbase]) end + return types +end - # merge the MROs - # this is a port of the merge() function at the bottom of: - # https://www.python.org/download/releases/2.3/mro/ - mro = Py[] - while !isempty(basemros) - # find the first head not contained in any tail - ok = false - b = PyNULL - for bmro in basemros - b = bmro[1] - if all(bmro -> !pyisin(b, bmro[2:end]), basemros) - ok = true - break - end - end - ok || error( - "Fatal inheritance error: could not merge MROs (mro=$mro, basemros=$basemros)", - ) - # add it to the list - push!(mro, b) - # remove it from consideration - for bmro in basemros - filter!(t -> !pyis(t, b), bmro) - end - # remove empty lists - filter!(x -> !isempty(x), basemros) - end - # check the original MRO is preserved - omro_ = filter(t -> pyisin(t, omro), mro) - @assert length(omro) == length(omro_) - @assert all(pyis(x, y) for (x, y) in zip(omro, omro_)) - - # get the names of the types in the MRO of pytype - xmro = [String[pyconvert_typename(t)] for t in mro] - - # add special names corresponding to certain interfaces - # these get inserted just above the topmost type satisfying the interface - for (t, x) in reverse(collect(zip(mro, xmro))) - if pyhasattr(t, "__array_struct__") - push!(x, "") - break - end - end - for (t, x) in reverse(collect(zip(mro, xmro))) - if pyhasattr(t, "__array_interface__") - push!(x, "") - break - end - end - for (t, x) in reverse(collect(zip(mro, xmro))) - if pyhasattr(t, "__array__") - push!(x, "") - break - end - end - for (t, x) in reverse(collect(zip(mro, xmro))) - if C.PyType_CheckBuffer(t) - push!(x, "") - break - end +function _pyconvert_get_rules(pytype::Py) + typemap = Dict{String,Py}() + tnames = Set{String}() + + function add_type!(t::Py) + tname = pyconvert_typename(t) + haskey(typemap, tname) || (typemap[tname] = t) + push!(tnames, tname) + return nothing end - # flatten to get the MRO as a list of strings - mro_strings = String[x for xs in xmro for x in xs] + for t in _pyconvert_collect_supertypes(pytype) + add_type!(t) + pyhasattr(t, "__array_struct__") && push!(tnames, "") + pyhasattr(t, "__array_interface__") && push!(tnames, "") + pyhasattr(t, "__array__") && push!(tnames, "") + (C.PyType_CheckBuffer(t) != 0) && push!(tnames, "") + end - typemap = Dict{String,Py}() - for t in mro - typemap[pyconvert_typename(t)] = t + for xtype in PYCONVERT_EXTRATYPES + pyissubclass(pytype, xtype) || continue + for t in _pyconvert_collect_supertypes(xtype) + add_type!(t) + end end - # get corresponding rules rules = PyConvertRuleInfo[ PyConvertRuleInfo( tname, @@ -245,10 +191,9 @@ function _pyconvert_get_rules(pytype::Py) rule.scope, rule.func, rule.order, - ) for tname in mro_strings for rule in get!(Vector{PyConvertRule}, PYCONVERT_RULES, tname) + ) for tname in tnames for rule in get!(Vector{PyConvertRule}, PYCONVERT_RULES, tname) ] - @debug "pyconvert" pytype mro = join(mro_strings, " ") return rules, typemap end From 56c3a12d8999830a4a940c950a5e574e5c702401 Mon Sep 17 00:00:00 2001 From: Christopher Rowley Date: Sun, 16 Nov 2025 23:13:55 +0000 Subject: [PATCH 06/11] Fix pyconvert supertype traversal and pandas test setup --- src/Convert/pyconvert.jl | 2 +- test/Compat.jl | 10 +++++++--- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/src/Convert/pyconvert.jl b/src/Convert/pyconvert.jl index 9ccfcb71..5ded6163 100644 --- a/src/Convert/pyconvert.jl +++ b/src/Convert/pyconvert.jl @@ -146,7 +146,7 @@ function _pyconvert_collect_supertypes(pytype::Py) types = Py[] while !isempty(queue) t = pop!(queue) - ptr = C.PyPtr(t) + ptr = getptr(t) ptr ∈ seen && continue push!(seen, ptr) push!(types, t) diff --git a/test/Compat.jl b/test/Compat.jl index ed230a71..407c5974 100644 --- a/test/Compat.jl +++ b/test/Compat.jl @@ -104,12 +104,16 @@ end end end -@testitem "Tables.jl" begin +@testitem "Tables.jl" setup=[Setup] begin @testset "pytable" begin x = (x = [1, 2, 3], y = ["a", "b", "c"]) # pandas - # TODO: install pandas and test properly - @test_throws PyException pytable(x, :pandas) + if Setup.devdeps + y = pytable(x, :pandas) + @test pyisinstance(y, pyimport("pandas").DataFrame) + else + @test_throws PyException pytable(x, :pandas) + end # columns y = pytable(x, :columns) @test pyeq(Bool, y, pydict(x = [1, 2, 3], y = ["a", "b", "c"])) From 7d98a71c3bc49b76735f40cc7af326140fc819bd Mon Sep 17 00:00:00 2001 From: Christopher Rowley Date: Mon, 17 Nov 2025 08:34:45 +0000 Subject: [PATCH 07/11] Centralize pyconvert rule registration --- src/Convert/Convert.jl | 5 +- src/Convert/ctypes.jl | 37 +++++--- src/Convert/numpy.jl | 98 +++++++++++-------- src/Convert/pandas.jl | 18 ++-- src/Convert/pyconvert.jl | 197 +++++++++++++++++++++------------------ src/JlWrap/JlWrap.jl | 1 + src/JlWrap/base.jl | 10 +- src/PythonCall.jl | 26 ++++++ src/Wrap/Wrap.jl | 65 +++++++------ test/Convert.jl | 14 +++ 10 files changed, 285 insertions(+), 186 deletions(-) diff --git a/src/Convert/Convert.jl b/src/Convert/Convert.jl index c3e7e0cd..7987a923 100644 --- a/src/Convert/Convert.jl +++ b/src/Convert/Convert.jl @@ -35,10 +35,7 @@ include("numpy.jl") include("pandas.jl") function __init__() - init_pyconvert() - init_ctypes() - init_numpy() - init_pandas() + init_pyconvert_extratypes() end end diff --git a/src/Convert/ctypes.jl b/src/Convert/ctypes.jl index e0a078c1..5310b9d8 100644 --- a/src/Convert/ctypes.jl +++ b/src/Convert/ctypes.jl @@ -34,7 +34,8 @@ const CTYPES_SIMPLE_TYPES = [ ("void_p", Ptr{Cvoid}), ] -function init_ctypes() +function ctypes_rule_specs() + specs = PyConvertRuleSpec[] for (t, T) in CTYPES_SIMPLE_TYPES isptr = endswith(t, "_p") isreal = !isptr @@ -47,18 +48,26 @@ function init_ctypes() rule = pyconvert_rule_ctypessimplevalue{T,false}() saferule = pyconvert_rule_ctypessimplevalue{T,true}() - t == "char_p" && pyconvert_add_rule(saferule, name, Cstring) - t == "wchar_p" && pyconvert_add_rule(saferule, name, Cwstring) - pyconvert_add_rule(saferule, name, T) - isuint && pyconvert_add_rule(sizeof(T) ≤ sizeof(UInt) ? saferule : rule, name, UInt) - isuint && pyconvert_add_rule(sizeof(T) < sizeof(Int) ? saferule : rule, name, Int) - isint && - !isuint && - pyconvert_add_rule(sizeof(T) ≤ sizeof(Int) ? saferule : rule, name, Int) - isint && pyconvert_add_rule(rule, name, Integer) - isfloat && pyconvert_add_rule(saferule, name, Float64) - isreal && pyconvert_add_rule(rule, name, Real) - isnumber && pyconvert_add_rule(rule, name, Number) - isptr && pyconvert_add_rule(saferule, name, Ptr) + t == "char_p" && push!(specs, (func = saferule, tname = name, type = Cstring, scope = Cstring)) + t == "wchar_p" && push!(specs, (func = saferule, tname = name, type = Cwstring, scope = Cwstring)) + push!(specs, (func = saferule, tname = name, type = T, scope = T)) + isuint && push!( + specs, + (func = sizeof(T) ≤ sizeof(UInt) ? saferule : rule, tname = name, type = UInt, scope = UInt), + ) + isuint && push!( + specs, + (func = sizeof(T) < sizeof(Int) ? saferule : rule, tname = name, type = Int, scope = Int), + ) + isint && !isuint && push!( + specs, + (func = sizeof(T) ≤ sizeof(Int) ? saferule : rule, tname = name, type = Int, scope = Int), + ) + isint && push!(specs, (func = rule, tname = name, type = Integer, scope = Integer)) + isfloat && push!(specs, (func = saferule, tname = name, type = Float64, scope = Float64)) + isreal && push!(specs, (func = rule, tname = name, type = Real, scope = Real)) + isnumber && push!(specs, (func = rule, tname = name, type = Number, scope = Number)) + isptr && push!(specs, (func = saferule, tname = name, type = Ptr, scope = Ptr)) end + return specs end diff --git a/src/Convert/numpy.jl b/src/Convert/numpy.jl index c73b0dce..24946b53 100644 --- a/src/Convert/numpy.jl +++ b/src/Convert/numpy.jl @@ -97,7 +97,8 @@ const NUMPY_SIMPLE_TYPES = [ ("complex128", ComplexF64), ] -function init_numpy() +function numpy_rule_specs() + specs = PyConvertRuleSpec[] # simple numeric scalar types for (t, T) in NUMPY_SIMPLE_TYPES isbool = occursin("bool", t) @@ -112,49 +113,72 @@ function init_numpy() rule = pyconvert_rule_numpysimplevalue{T,false}() saferule = pyconvert_rule_numpysimplevalue{T,true}() - pyconvert_add_rule(saferule, name, T, Any) - isuint && pyconvert_add_rule(sizeof(T) ≤ sizeof(UInt) ? saferule : rule, name, UInt) - isuint && pyconvert_add_rule(sizeof(T) < sizeof(Int) ? saferule : rule, name, Int) - isint && - !isuint && - pyconvert_add_rule(sizeof(T) ≤ sizeof(Int) ? saferule : rule, name, Int) - isint && pyconvert_add_rule(rule, name, Integer) - isfloat && pyconvert_add_rule(saferule, name, Float64) - isreal && pyconvert_add_rule(rule, name, Real) - iscomplex && pyconvert_add_rule(saferule, name, ComplexF64) - iscomplex && pyconvert_add_rule(rule, name, Complex) - isnumber && pyconvert_add_rule(rule, name, Number) + push!(specs, (func = saferule, tname = name, type = T, scope = Any)) + isuint && push!( + specs, + ( + func = sizeof(T) ≤ sizeof(UInt) ? saferule : rule, + tname = name, + type = UInt, + scope = UInt, + ), + ) + isuint && push!( + specs, + ( + func = sizeof(T) < sizeof(Int) ? saferule : rule, + tname = name, + type = Int, + scope = Int, + ), + ) + isint && !isuint && push!( + specs, + (func = sizeof(T) ≤ sizeof(Int) ? saferule : rule, tname = name, type = Int, scope = Int), + ) + isint && push!(specs, (func = rule, tname = name, type = Integer, scope = Integer)) + isfloat && push!(specs, (func = saferule, tname = name, type = Float64, scope = Float64)) + isreal && push!(specs, (func = rule, tname = name, type = Real, scope = Real)) + iscomplex && push!(specs, (func = saferule, tname = name, type = ComplexF64, scope = ComplexF64)) + iscomplex && push!(specs, (func = rule, tname = name, type = Complex, scope = Complex)) + isnumber && push!(specs, (func = rule, tname = name, type = Number, scope = Number)) end # datetime64 - pyconvert_add_rule( - pyconvert_rule_datetime64, - "numpy:datetime64", - DateTime64, - Any, + push!( + specs, + (func = pyconvert_rule_datetime64, tname = "numpy:datetime64", type = DateTime64, scope = Any), ) - pyconvert_add_rule(pyconvert_rule_datetime64, "numpy:datetime64", InlineDateTime64) - pyconvert_add_rule( - pyconvert_rule_datetime64, - "numpy:datetime64", - NumpyDates.DatesInstant, + push!(specs, (func = pyconvert_rule_datetime64, tname = "numpy:datetime64", type = InlineDateTime64, scope = InlineDateTime64)) + push!( + specs, + ( + func = pyconvert_rule_datetime64, + tname = "numpy:datetime64", + type = NumpyDates.DatesInstant, + scope = NumpyDates.DatesInstant, + ), ) - pyconvert_add_rule(pyconvert_rule_datetime64, "numpy:datetime64", Missing, Missing) - pyconvert_add_rule(pyconvert_rule_datetime64, "numpy:datetime64", Nothing, Nothing) + push!(specs, (func = pyconvert_rule_datetime64, tname = "numpy:datetime64", type = Missing, scope = Missing)) + push!(specs, (func = pyconvert_rule_datetime64, tname = "numpy:datetime64", type = Nothing, scope = Nothing)) # timedelta64 - pyconvert_add_rule( - pyconvert_rule_timedelta64, - "numpy:timedelta64", - TimeDelta64, - Any, + push!( + specs, + (func = pyconvert_rule_timedelta64, tname = "numpy:timedelta64", type = TimeDelta64, scope = Any), ) - pyconvert_add_rule(pyconvert_rule_timedelta64, "numpy:timedelta64", InlineTimeDelta64) - pyconvert_add_rule( - pyconvert_rule_timedelta64, - "numpy:timedelta64", - NumpyDates.DatesPeriod, + push!(specs, (func = pyconvert_rule_timedelta64, tname = "numpy:timedelta64", type = InlineTimeDelta64, scope = InlineTimeDelta64)) + push!( + specs, + ( + func = pyconvert_rule_timedelta64, + tname = "numpy:timedelta64", + type = NumpyDates.DatesPeriod, + scope = NumpyDates.DatesPeriod, + ), ) - pyconvert_add_rule(pyconvert_rule_timedelta64, "numpy:timedelta64", Missing, Missing) - pyconvert_add_rule(pyconvert_rule_timedelta64, "numpy:timedelta64", Nothing, Nothing) + push!(specs, (func = pyconvert_rule_timedelta64, tname = "numpy:timedelta64", type = Missing, scope = Missing)) + push!(specs, (func = pyconvert_rule_timedelta64, tname = "numpy:timedelta64", type = Nothing, scope = Nothing)) + + return specs end diff --git a/src/Convert/pandas.jl b/src/Convert/pandas.jl index 1604979e..13bdde75 100644 --- a/src/Convert/pandas.jl +++ b/src/Convert/pandas.jl @@ -1,12 +1,14 @@ pyconvert_rule_pandas_na(::Type{Nothing}, x::Py) = pyconvert_return(nothing) pyconvert_rule_pandas_na(::Type{Missing}, x::Py) = pyconvert_return(missing) -function init_pandas() - pyconvert_add_rule( - pyconvert_rule_pandas_na, - "pandas._libs.missing:NAType", - Missing, - Any, - ) - pyconvert_add_rule(pyconvert_rule_pandas_na, "pandas._libs.missing:NAType", Nothing, Nothing) +function pandas_rule_specs() + return PyConvertRuleSpec[ + (func = pyconvert_rule_pandas_na, tname = "pandas._libs.missing:NAType", type = Missing, scope = Any), + ( + func = pyconvert_rule_pandas_na, + tname = "pandas._libs.missing:NAType", + type = Nothing, + scope = Nothing, + ), + ] end diff --git a/src/Convert/pyconvert.jl b/src/Convert/pyconvert.jl index 5ded6163..92325d86 100644 --- a/src/Convert/pyconvert.jl +++ b/src/Convert/pyconvert.jl @@ -9,6 +9,7 @@ end const PYCONVERT_RULES = Dict{String,Vector{PyConvertRule}}() const PYCONVERT_RULE_ORDER = Ref{Int}(0) const PYCONVERT_EXTRATYPES = Py[] +const PyConvertRuleSpec = NamedTuple{(:func, :tname, :type, :scope),Tuple{Function,String,Type,Type}} """ pyconvert_add_rule(func::Function, tname::String, ::Type{T}, ::Type{S}=T) where {T,S} @@ -422,7 +423,8 @@ pyconvertarg(::Type{T}, x, name) where {T} = @autopy x @pyconvert T x_ begin pythrow() end -function init_pyconvert() +function init_pyconvert_extratypes() + isempty(PYCONVERT_EXTRATYPES) || return push!(PYCONVERT_EXTRATYPES, pyimport("io" => "IOBase")) push!( PYCONVERT_EXTRATYPES, @@ -432,93 +434,110 @@ function init_pyconvert() PYCONVERT_EXTRATYPES, pyimport("collections.abc" => ("Iterable", "Sequence", "Set", "Mapping"))..., ) +end - pyconvert_add_rule(pyconvert_rule_none, "builtins:NoneType", Nothing, Any) - pyconvert_add_rule(pyconvert_rule_bool, "builtins:bool", Bool, Any) - pyconvert_add_rule(pyconvert_rule_float, "builtins:float", Float64, Any) - pyconvert_add_rule( - pyconvert_rule_complex, - "builtins:complex", - Complex{Float64}, - Any, - ) - pyconvert_add_rule(pyconvert_rule_int, "numbers:Integral", Integer, Any) - pyconvert_add_rule(pyconvert_rule_str, "builtins:str", String, Any) - pyconvert_add_rule( - pyconvert_rule_bytes, - "builtins:bytes", - Base.CodeUnits{UInt8,String}, - Any, - ) - pyconvert_add_rule( - pyconvert_rule_range, - "builtins:range", - StepRange{<:Integer,<:Integer}, - Any, - ) - pyconvert_add_rule( - pyconvert_rule_fraction, - "numbers:Rational", - Rational{<:Integer}, - Any, - ) - pyconvert_add_rule(pyconvert_rule_iterable, "builtins:tuple", NamedTuple, Any) - pyconvert_add_rule(pyconvert_rule_iterable, "builtins:tuple", Tuple, Any) - pyconvert_add_rule(pyconvert_rule_datetime, "datetime:datetime", DateTime, Any) - pyconvert_add_rule(pyconvert_rule_date, "datetime:date", Date, Any) - pyconvert_add_rule(pyconvert_rule_time, "datetime:time", Time, Any) - pyconvert_add_rule( - pyconvert_rule_timedelta, - "datetime:timedelta", - Microsecond, - Any, - ) - pyconvert_add_rule( - pyconvert_rule_exception, - "builtins:BaseException", - PyException, - Any, - ) - - pyconvert_add_rule(pyconvert_rule_none, "builtins:NoneType", Missing, Missing) - pyconvert_add_rule(pyconvert_rule_bool, "builtins:bool", Number, Number) - pyconvert_add_rule(pyconvert_rule_float, "numbers:Real", Number, Number) - pyconvert_add_rule(pyconvert_rule_float, "builtins:float", Nothing, Nothing) - pyconvert_add_rule(pyconvert_rule_float, "builtins:float", Missing, Missing) - pyconvert_add_rule(pyconvert_rule_complex, "numbers:Complex", Number, Number) - pyconvert_add_rule(pyconvert_rule_int, "numbers:Integral", Number, Number) - pyconvert_add_rule(pyconvert_rule_str, "builtins:str", Symbol) - pyconvert_add_rule(pyconvert_rule_str, "builtins:str", Char) - pyconvert_add_rule(pyconvert_rule_bytes, "builtins:bytes", Vector{UInt8}) - pyconvert_add_rule( - pyconvert_rule_range, - "builtins:range", - UnitRange{<:Integer}, - ) - pyconvert_add_rule(pyconvert_rule_fraction, "numbers:Rational", Number, Number) - pyconvert_add_rule( - pyconvert_rule_iterable, - "collections.abc:Iterable", - Vector, - ) - pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", Tuple) - pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", Pair) - pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", Set) - pyconvert_add_rule( - pyconvert_rule_iterable, - "collections.abc:Sequence", - Vector, - ) - pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Sequence", Tuple) - pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Set", Set) - pyconvert_add_rule(pyconvert_rule_mapping, "collections.abc:Mapping", Dict) - pyconvert_add_rule( - pyconvert_rule_timedelta, - "datetime:timedelta", - Millisecond, - ) - pyconvert_add_rule(pyconvert_rule_timedelta, "datetime:timedelta", Second) - pyconvert_add_rule(pyconvert_rule_timedelta, "datetime:timedelta", Nanosecond) - - pyconvert_add_rule(pyconvert_rule_object, "builtins:object", Py, Any) +function pyconvert_rule_specs() + return PyConvertRuleSpec[ + (func = pyconvert_rule_none, tname = "builtins:NoneType", type = Nothing, scope = Any), + (func = pyconvert_rule_bool, tname = "builtins:bool", type = Bool, scope = Any), + (func = pyconvert_rule_float, tname = "builtins:float", type = Float64, scope = Any), + ( + func = pyconvert_rule_complex, + tname = "builtins:complex", + type = Complex{Float64}, + scope = Any, + ), + (func = pyconvert_rule_int, tname = "numbers:Integral", type = Integer, scope = Any), + (func = pyconvert_rule_str, tname = "builtins:str", type = String, scope = Any), + ( + func = pyconvert_rule_bytes, + tname = "builtins:bytes", + type = Base.CodeUnits{UInt8,String}, + scope = Any, + ), + ( + func = pyconvert_rule_range, + tname = "builtins:range", + type = StepRange{<:Integer,<:Integer}, + scope = Any, + ), + ( + func = pyconvert_rule_fraction, + tname = "numbers:Rational", + type = Rational{<:Integer}, + scope = Any, + ), + (func = pyconvert_rule_iterable, tname = "builtins:tuple", type = NamedTuple, scope = Any), + (func = pyconvert_rule_iterable, tname = "builtins:tuple", type = Tuple, scope = Any), + (func = pyconvert_rule_datetime, tname = "datetime:datetime", type = DateTime, scope = Any), + (func = pyconvert_rule_date, tname = "datetime:date", type = Date, scope = Any), + (func = pyconvert_rule_time, tname = "datetime:time", type = Time, scope = Any), + ( + func = pyconvert_rule_timedelta, + tname = "datetime:timedelta", + type = Microsecond, + scope = Any, + ), + ( + func = pyconvert_rule_exception, + tname = "builtins:BaseException", + type = PyException, + scope = Any, + ), + (func = pyconvert_rule_none, tname = "builtins:NoneType", type = Missing, scope = Missing), + (func = pyconvert_rule_bool, tname = "builtins:bool", type = Number, scope = Number), + (func = pyconvert_rule_float, tname = "numbers:Real", type = Number, scope = Number), + (func = pyconvert_rule_float, tname = "builtins:float", type = Nothing, scope = Nothing), + (func = pyconvert_rule_float, tname = "builtins:float", type = Missing, scope = Missing), + (func = pyconvert_rule_complex, tname = "numbers:Complex", type = Number, scope = Number), + (func = pyconvert_rule_int, tname = "numbers:Integral", type = Number, scope = Number), + (func = pyconvert_rule_str, tname = "builtins:str", type = Symbol, scope = Symbol), + (func = pyconvert_rule_str, tname = "builtins:str", type = Char, scope = Char), + (func = pyconvert_rule_bytes, tname = "builtins:bytes", type = Vector{UInt8}, scope = Vector{UInt8}), + ( + func = pyconvert_rule_range, + tname = "builtins:range", + type = UnitRange{<:Integer}, + scope = UnitRange{<:Integer}, + ), + (func = pyconvert_rule_fraction, tname = "numbers:Rational", type = Number, scope = Number), + ( + func = pyconvert_rule_iterable, + tname = "collections.abc:Iterable", + type = Vector, + scope = Vector, + ), + (func = pyconvert_rule_iterable, tname = "collections.abc:Iterable", type = Tuple, scope = Tuple), + (func = pyconvert_rule_iterable, tname = "collections.abc:Iterable", type = Pair, scope = Pair), + (func = pyconvert_rule_iterable, tname = "collections.abc:Iterable", type = Set, scope = Set), + ( + func = pyconvert_rule_iterable, + tname = "collections.abc:Sequence", + type = Vector, + scope = Vector, + ), + (func = pyconvert_rule_iterable, tname = "collections.abc:Sequence", type = Tuple, scope = Tuple), + (func = pyconvert_rule_iterable, tname = "collections.abc:Set", type = Set, scope = Set), + (func = pyconvert_rule_mapping, tname = "collections.abc:Mapping", type = Dict, scope = Dict), + ( + func = pyconvert_rule_timedelta, + tname = "datetime:timedelta", + type = Millisecond, + scope = Millisecond, + ), + (func = pyconvert_rule_timedelta, tname = "datetime:timedelta", type = Second, scope = Second), + ( + func = pyconvert_rule_timedelta, + tname = "datetime:timedelta", + type = Nanosecond, + scope = Nanosecond, + ), + ] end + +pyconvert_fallback_rule_specs() = PyConvertRuleSpec[( + func = pyconvert_rule_object, + tname = "builtins:object", + type = Py, + scope = Any, +)] diff --git a/src/JlWrap/JlWrap.jl b/src/JlWrap/JlWrap.jl index 2ace6b85..763dc716 100644 --- a/src/JlWrap/JlWrap.jl +++ b/src/JlWrap/JlWrap.jl @@ -11,6 +11,7 @@ using ..NumpyDates: NumpyDates using ..C using ..Core using ..Convert +using ..Convert: PyConvertRuleSpec using ..GC: GC using ..GIL diff --git a/src/JlWrap/base.jl b/src/JlWrap/base.jl index cfd9e1bb..6f5e3063 100644 --- a/src/JlWrap/base.jl +++ b/src/JlWrap/base.jl @@ -23,11 +23,15 @@ pyjlvalue(x) = @autopy x _pyjl_getvalue(x_) function init_base() setptr!(pyjlbasetype, incref(Cjl.PyJuliaBase_Type[])) pyjuliacallmodule.JlBase = pyjlbasetype - - # conversion rule - pyconvert_add_rule(pyconvert_rule_jlvalue, "juliacall:JlBase", Any, Any) end +jlwrap_rule_specs() = PyConvertRuleSpec[( + func = pyconvert_rule_jlvalue, + tname = "juliacall:JlBase", + type = Any, + scope = Any, +)] + pyconvert_rule_jlvalue(::Type{T}, x::Py) where {T} = pyconvert_tryconvert(T, _pyjl_getvalue(x)) diff --git a/src/PythonCall.jl b/src/PythonCall.jl index e8c7597f..eca00571 100644 --- a/src/PythonCall.jl +++ b/src/PythonCall.jl @@ -31,4 +31,30 @@ for k in [ @eval using .JlWrap: $k end +function __init__() + Convert.init_pyconvert_extratypes() + + for rule in Convert.pyconvert_rule_specs() + pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) + end + for rule in Convert.ctypes_rule_specs() + pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) + end + for rule in Convert.numpy_rule_specs() + pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) + end + for rule in Convert.pandas_rule_specs() + pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) + end + for rule in Wrap.wrap_pyconvert_rule_specs() + pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) + end + for rule in JlWrap.jlwrap_rule_specs() + pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) + end + for rule in Convert.pyconvert_fallback_rule_specs() + pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) + end +end + end diff --git a/src/Wrap/Wrap.jl b/src/Wrap/Wrap.jl index 182f547c..eacd562c 100644 --- a/src/Wrap/Wrap.jl +++ b/src/Wrap/Wrap.jl @@ -11,6 +11,7 @@ using ..NumpyDates using ..C using ..Core using ..Convert +using ..Convert: PyConvertRuleSpec using ..PyMacro import ..PythonCall: @@ -31,37 +32,39 @@ include("PyIO.jl") include("PyTable.jl") include("PyPandasDataFrame.jl") -function __init__() - pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) - pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) - pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) - pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) - - pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", PyIterable) - pyconvert_add_rule(pyconvert_rule_sequence, "collections.abc:Sequence", PyList) - pyconvert_add_rule(pyconvert_rule_set, "collections.abc:Set", PySet) - pyconvert_add_rule(pyconvert_rule_mapping, "collections.abc:Mapping", PyDict) - pyconvert_add_rule(pyconvert_rule_io, "io:IOBase", PyIO) - pyconvert_add_rule(pyconvert_rule_io, "_io:_IOBase", PyIO) - pyconvert_add_rule( - pyconvert_rule_pandasdataframe, - "pandas.core.frame:DataFrame", - PyPandasDataFrame, - ) - pyconvert_add_rule( - pyconvert_rule_sequence, - "pandas.core.arrays.base:ExtensionArray", - PyList, - ) - - pyconvert_add_rule(pyconvert_rule_array, "", Array) - pyconvert_add_rule(pyconvert_rule_array, "", Array) - pyconvert_add_rule(pyconvert_rule_array, "", Array) - pyconvert_add_rule(pyconvert_rule_array, "", Array) - pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray) - pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray) - pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray) - pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray) +function wrap_pyconvert_rule_specs() + return PyConvertRuleSpec[ + (func = pyconvert_rule_array_nocopy, tname = "", type = PyArray, scope = Any), + (func = pyconvert_rule_array_nocopy, tname = "", type = PyArray, scope = Any), + (func = pyconvert_rule_array_nocopy, tname = "", type = PyArray, scope = Any), + (func = pyconvert_rule_array_nocopy, tname = "", type = PyArray, scope = Any), + (func = pyconvert_rule_iterable, tname = "collections.abc:Iterable", type = PyIterable, scope = PyIterable), + (func = pyconvert_rule_sequence, tname = "collections.abc:Sequence", type = PyList, scope = PyList), + (func = pyconvert_rule_set, tname = "collections.abc:Set", type = PySet, scope = PySet), + (func = pyconvert_rule_mapping, tname = "collections.abc:Mapping", type = PyDict, scope = PyDict), + (func = pyconvert_rule_io, tname = "io:IOBase", type = PyIO, scope = PyIO), + (func = pyconvert_rule_io, tname = "_io:_IOBase", type = PyIO, scope = PyIO), + ( + func = pyconvert_rule_pandasdataframe, + tname = "pandas.core.frame:DataFrame", + type = PyPandasDataFrame, + scope = PyPandasDataFrame, + ), + ( + func = pyconvert_rule_sequence, + tname = "pandas.core.arrays.base:ExtensionArray", + type = PyList, + scope = PyList, + ), + (func = pyconvert_rule_array, tname = "", type = Array, scope = Array), + (func = pyconvert_rule_array, tname = "", type = Array, scope = Array), + (func = pyconvert_rule_array, tname = "", type = Array, scope = Array), + (func = pyconvert_rule_array, tname = "", type = Array, scope = Array), + (func = pyconvert_rule_array, tname = "", type = AbstractArray, scope = AbstractArray), + (func = pyconvert_rule_array, tname = "", type = AbstractArray, scope = AbstractArray), + (func = pyconvert_rule_array, tname = "", type = AbstractArray, scope = AbstractArray), + (func = pyconvert_rule_array, tname = "", type = AbstractArray, scope = AbstractArray), + ] end end diff --git a/test/Convert.jl b/test/Convert.jl index d3404660..d81d4787 100644 --- a/test/Convert.jl +++ b/test/Convert.jl @@ -286,6 +286,20 @@ end end end +@testitem "numpy array → PyArray" setup=[Setup] begin + if Setup.devdeps + np = pyimport("numpy") + + arr = np.array([1, 2, 3], dtype = np.int64) + pyarr = pyconvert(PyArray, arr) + + @test pyarr isa PyArray + @test collect(pyarr) == [1, 2, 3] + else + @test_skip Setup.devdeps + end +end + @testitem "pandas NA" setup=[Setup] begin if Setup.devdeps pd = pyimport("pandas") From 5524df0a2e0ac3fd601cd908dcee25a6a0236a73 Mon Sep 17 00:00:00 2001 From: Christopher Rowley Date: Mon, 17 Nov 2025 09:02:53 +0000 Subject: [PATCH 08/11] Make pyconvert registration explicit --- src/Convert/ctypes.jl | 6 ++ src/Convert/numpy.jl | 6 ++ src/Convert/pandas.jl | 6 ++ src/Convert/pyconvert.jl | 146 ++++++++++++--------------------------- src/JlWrap/JlWrap.jl | 1 - src/JlWrap/base.jl | 9 +-- src/PythonCall.jl | 28 ++------ src/Wrap/Wrap.jl | 55 ++++++--------- 8 files changed, 92 insertions(+), 165 deletions(-) diff --git a/src/Convert/ctypes.jl b/src/Convert/ctypes.jl index 5310b9d8..89ee9975 100644 --- a/src/Convert/ctypes.jl +++ b/src/Convert/ctypes.jl @@ -71,3 +71,9 @@ function ctypes_rule_specs() end return specs end + +function register_ctypes_rules!() + for rule in ctypes_rule_specs() + pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) + end +end diff --git a/src/Convert/numpy.jl b/src/Convert/numpy.jl index 24946b53..b4e95059 100644 --- a/src/Convert/numpy.jl +++ b/src/Convert/numpy.jl @@ -182,3 +182,9 @@ function numpy_rule_specs() return specs end + +function register_numpy_rules!() + for rule in numpy_rule_specs() + pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) + end +end diff --git a/src/Convert/pandas.jl b/src/Convert/pandas.jl index 13bdde75..5413fe6c 100644 --- a/src/Convert/pandas.jl +++ b/src/Convert/pandas.jl @@ -12,3 +12,9 @@ function pandas_rule_specs() ), ] end + +function register_pandas_rules!() + for rule in pandas_rule_specs() + pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) + end +end diff --git a/src/Convert/pyconvert.jl b/src/Convert/pyconvert.jl index 92325d86..0d976c09 100644 --- a/src/Convert/pyconvert.jl +++ b/src/Convert/pyconvert.jl @@ -436,108 +436,48 @@ function init_pyconvert_extratypes() ) end -function pyconvert_rule_specs() - return PyConvertRuleSpec[ - (func = pyconvert_rule_none, tname = "builtins:NoneType", type = Nothing, scope = Any), - (func = pyconvert_rule_bool, tname = "builtins:bool", type = Bool, scope = Any), - (func = pyconvert_rule_float, tname = "builtins:float", type = Float64, scope = Any), - ( - func = pyconvert_rule_complex, - tname = "builtins:complex", - type = Complex{Float64}, - scope = Any, - ), - (func = pyconvert_rule_int, tname = "numbers:Integral", type = Integer, scope = Any), - (func = pyconvert_rule_str, tname = "builtins:str", type = String, scope = Any), - ( - func = pyconvert_rule_bytes, - tname = "builtins:bytes", - type = Base.CodeUnits{UInt8,String}, - scope = Any, - ), - ( - func = pyconvert_rule_range, - tname = "builtins:range", - type = StepRange{<:Integer,<:Integer}, - scope = Any, - ), - ( - func = pyconvert_rule_fraction, - tname = "numbers:Rational", - type = Rational{<:Integer}, - scope = Any, - ), - (func = pyconvert_rule_iterable, tname = "builtins:tuple", type = NamedTuple, scope = Any), - (func = pyconvert_rule_iterable, tname = "builtins:tuple", type = Tuple, scope = Any), - (func = pyconvert_rule_datetime, tname = "datetime:datetime", type = DateTime, scope = Any), - (func = pyconvert_rule_date, tname = "datetime:date", type = Date, scope = Any), - (func = pyconvert_rule_time, tname = "datetime:time", type = Time, scope = Any), - ( - func = pyconvert_rule_timedelta, - tname = "datetime:timedelta", - type = Microsecond, - scope = Any, - ), - ( - func = pyconvert_rule_exception, - tname = "builtins:BaseException", - type = PyException, - scope = Any, - ), - (func = pyconvert_rule_none, tname = "builtins:NoneType", type = Missing, scope = Missing), - (func = pyconvert_rule_bool, tname = "builtins:bool", type = Number, scope = Number), - (func = pyconvert_rule_float, tname = "numbers:Real", type = Number, scope = Number), - (func = pyconvert_rule_float, tname = "builtins:float", type = Nothing, scope = Nothing), - (func = pyconvert_rule_float, tname = "builtins:float", type = Missing, scope = Missing), - (func = pyconvert_rule_complex, tname = "numbers:Complex", type = Number, scope = Number), - (func = pyconvert_rule_int, tname = "numbers:Integral", type = Number, scope = Number), - (func = pyconvert_rule_str, tname = "builtins:str", type = Symbol, scope = Symbol), - (func = pyconvert_rule_str, tname = "builtins:str", type = Char, scope = Char), - (func = pyconvert_rule_bytes, tname = "builtins:bytes", type = Vector{UInt8}, scope = Vector{UInt8}), - ( - func = pyconvert_rule_range, - tname = "builtins:range", - type = UnitRange{<:Integer}, - scope = UnitRange{<:Integer}, - ), - (func = pyconvert_rule_fraction, tname = "numbers:Rational", type = Number, scope = Number), - ( - func = pyconvert_rule_iterable, - tname = "collections.abc:Iterable", - type = Vector, - scope = Vector, - ), - (func = pyconvert_rule_iterable, tname = "collections.abc:Iterable", type = Tuple, scope = Tuple), - (func = pyconvert_rule_iterable, tname = "collections.abc:Iterable", type = Pair, scope = Pair), - (func = pyconvert_rule_iterable, tname = "collections.abc:Iterable", type = Set, scope = Set), - ( - func = pyconvert_rule_iterable, - tname = "collections.abc:Sequence", - type = Vector, - scope = Vector, - ), - (func = pyconvert_rule_iterable, tname = "collections.abc:Sequence", type = Tuple, scope = Tuple), - (func = pyconvert_rule_iterable, tname = "collections.abc:Set", type = Set, scope = Set), - (func = pyconvert_rule_mapping, tname = "collections.abc:Mapping", type = Dict, scope = Dict), - ( - func = pyconvert_rule_timedelta, - tname = "datetime:timedelta", - type = Millisecond, - scope = Millisecond, - ), - (func = pyconvert_rule_timedelta, tname = "datetime:timedelta", type = Second, scope = Second), - ( - func = pyconvert_rule_timedelta, - tname = "datetime:timedelta", - type = Nanosecond, - scope = Nanosecond, - ), - ] +function register_pyconvert_rules!() + pyconvert_add_rule(pyconvert_rule_none, "builtins:NoneType", Nothing, Any) + pyconvert_add_rule(pyconvert_rule_bool, "builtins:bool", Bool, Any) + pyconvert_add_rule(pyconvert_rule_float, "builtins:float", Float64, Any) + pyconvert_add_rule(pyconvert_rule_complex, "builtins:complex", Complex{Float64}, Any) + pyconvert_add_rule(pyconvert_rule_int, "numbers:Integral", Integer, Any) + pyconvert_add_rule(pyconvert_rule_str, "builtins:str", String, Any) + pyconvert_add_rule(pyconvert_rule_bytes, "builtins:bytes", Base.CodeUnits{UInt8,String}, Any) + pyconvert_add_rule(pyconvert_rule_range, "builtins:range", StepRange{<:Integer,<:Integer}, Any) + pyconvert_add_rule(pyconvert_rule_fraction, "numbers:Rational", Rational{<:Integer}, Any) + pyconvert_add_rule(pyconvert_rule_iterable, "builtins:tuple", NamedTuple, Any) + pyconvert_add_rule(pyconvert_rule_iterable, "builtins:tuple", Tuple, Any) + pyconvert_add_rule(pyconvert_rule_datetime, "datetime:datetime", DateTime, Any) + pyconvert_add_rule(pyconvert_rule_date, "datetime:date", Date, Any) + pyconvert_add_rule(pyconvert_rule_time, "datetime:time", Time, Any) + pyconvert_add_rule(pyconvert_rule_timedelta, "datetime:timedelta", Microsecond, Any) + pyconvert_add_rule(pyconvert_rule_exception, "builtins:BaseException", PyException, Any) + pyconvert_add_rule(pyconvert_rule_none, "builtins:NoneType", Missing, Missing) + pyconvert_add_rule(pyconvert_rule_bool, "builtins:bool", Number, Number) + pyconvert_add_rule(pyconvert_rule_float, "numbers:Real", Number, Number) + pyconvert_add_rule(pyconvert_rule_float, "builtins:float", Nothing, Nothing) + pyconvert_add_rule(pyconvert_rule_float, "builtins:float", Missing, Missing) + pyconvert_add_rule(pyconvert_rule_complex, "numbers:Complex", Number, Number) + pyconvert_add_rule(pyconvert_rule_int, "numbers:Integral", Number, Number) + pyconvert_add_rule(pyconvert_rule_str, "builtins:str", Symbol, Symbol) + pyconvert_add_rule(pyconvert_rule_str, "builtins:str", Char, Char) + pyconvert_add_rule(pyconvert_rule_bytes, "builtins:bytes", Vector{UInt8}, Vector{UInt8}) + pyconvert_add_rule(pyconvert_rule_range, "builtins:range", UnitRange{<:Integer}, UnitRange{<:Integer}) + pyconvert_add_rule(pyconvert_rule_fraction, "numbers:Rational", Number, Number) + pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", Vector, Vector) + pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", Tuple, Tuple) + pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", Pair, Pair) + pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", Set, Set) + pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Sequence", Vector, Vector) + pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Sequence", Tuple, Tuple) + pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Set", Set, Set) + pyconvert_add_rule(pyconvert_rule_mapping, "collections.abc:Mapping", Dict, Dict) + pyconvert_add_rule(pyconvert_rule_timedelta, "datetime:timedelta", Millisecond, Millisecond) + pyconvert_add_rule(pyconvert_rule_timedelta, "datetime:timedelta", Second, Second) + pyconvert_add_rule(pyconvert_rule_timedelta, "datetime:timedelta", Nanosecond, Nanosecond) end -pyconvert_fallback_rule_specs() = PyConvertRuleSpec[( - func = pyconvert_rule_object, - tname = "builtins:object", - type = Py, - scope = Any, -)] +function register_pyconvert_fallback_rules!() + pyconvert_add_rule(pyconvert_rule_object, "builtins:object", Py, Any) +end diff --git a/src/JlWrap/JlWrap.jl b/src/JlWrap/JlWrap.jl index 763dc716..2ace6b85 100644 --- a/src/JlWrap/JlWrap.jl +++ b/src/JlWrap/JlWrap.jl @@ -11,7 +11,6 @@ using ..NumpyDates: NumpyDates using ..C using ..Core using ..Convert -using ..Convert: PyConvertRuleSpec using ..GC: GC using ..GIL diff --git a/src/JlWrap/base.jl b/src/JlWrap/base.jl index 6f5e3063..d82581b1 100644 --- a/src/JlWrap/base.jl +++ b/src/JlWrap/base.jl @@ -25,12 +25,9 @@ function init_base() pyjuliacallmodule.JlBase = pyjlbasetype end -jlwrap_rule_specs() = PyConvertRuleSpec[( - func = pyconvert_rule_jlvalue, - tname = "juliacall:JlBase", - type = Any, - scope = Any, -)] +function register_jlwrap_rules!() + pyconvert_add_rule(pyconvert_rule_jlvalue, "juliacall:JlBase", Any, Any) +end pyconvert_rule_jlvalue(::Type{T}, x::Py) where {T} = pyconvert_tryconvert(T, _pyjl_getvalue(x)) diff --git a/src/PythonCall.jl b/src/PythonCall.jl index eca00571..eb632c2c 100644 --- a/src/PythonCall.jl +++ b/src/PythonCall.jl @@ -34,27 +34,13 @@ end function __init__() Convert.init_pyconvert_extratypes() - for rule in Convert.pyconvert_rule_specs() - pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) - end - for rule in Convert.ctypes_rule_specs() - pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) - end - for rule in Convert.numpy_rule_specs() - pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) - end - for rule in Convert.pandas_rule_specs() - pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) - end - for rule in Wrap.wrap_pyconvert_rule_specs() - pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) - end - for rule in JlWrap.jlwrap_rule_specs() - pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) - end - for rule in Convert.pyconvert_fallback_rule_specs() - pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) - end + Convert.register_pyconvert_rules!() + Convert.register_ctypes_rules!() + Convert.register_numpy_rules!() + Convert.register_pandas_rules!() + Wrap.register_wrap_pyconvert_rules!() + JlWrap.register_jlwrap_rules!() + Convert.register_pyconvert_fallback_rules!() end end diff --git a/src/Wrap/Wrap.jl b/src/Wrap/Wrap.jl index eacd562c..e9a667c8 100644 --- a/src/Wrap/Wrap.jl +++ b/src/Wrap/Wrap.jl @@ -11,7 +11,6 @@ using ..NumpyDates using ..C using ..Core using ..Convert -using ..Convert: PyConvertRuleSpec using ..PyMacro import ..PythonCall: @@ -32,39 +31,27 @@ include("PyIO.jl") include("PyTable.jl") include("PyPandasDataFrame.jl") -function wrap_pyconvert_rule_specs() - return PyConvertRuleSpec[ - (func = pyconvert_rule_array_nocopy, tname = "", type = PyArray, scope = Any), - (func = pyconvert_rule_array_nocopy, tname = "", type = PyArray, scope = Any), - (func = pyconvert_rule_array_nocopy, tname = "", type = PyArray, scope = Any), - (func = pyconvert_rule_array_nocopy, tname = "", type = PyArray, scope = Any), - (func = pyconvert_rule_iterable, tname = "collections.abc:Iterable", type = PyIterable, scope = PyIterable), - (func = pyconvert_rule_sequence, tname = "collections.abc:Sequence", type = PyList, scope = PyList), - (func = pyconvert_rule_set, tname = "collections.abc:Set", type = PySet, scope = PySet), - (func = pyconvert_rule_mapping, tname = "collections.abc:Mapping", type = PyDict, scope = PyDict), - (func = pyconvert_rule_io, tname = "io:IOBase", type = PyIO, scope = PyIO), - (func = pyconvert_rule_io, tname = "_io:_IOBase", type = PyIO, scope = PyIO), - ( - func = pyconvert_rule_pandasdataframe, - tname = "pandas.core.frame:DataFrame", - type = PyPandasDataFrame, - scope = PyPandasDataFrame, - ), - ( - func = pyconvert_rule_sequence, - tname = "pandas.core.arrays.base:ExtensionArray", - type = PyList, - scope = PyList, - ), - (func = pyconvert_rule_array, tname = "", type = Array, scope = Array), - (func = pyconvert_rule_array, tname = "", type = Array, scope = Array), - (func = pyconvert_rule_array, tname = "", type = Array, scope = Array), - (func = pyconvert_rule_array, tname = "", type = Array, scope = Array), - (func = pyconvert_rule_array, tname = "", type = AbstractArray, scope = AbstractArray), - (func = pyconvert_rule_array, tname = "", type = AbstractArray, scope = AbstractArray), - (func = pyconvert_rule_array, tname = "", type = AbstractArray, scope = AbstractArray), - (func = pyconvert_rule_array, tname = "", type = AbstractArray, scope = AbstractArray), - ] +function register_wrap_pyconvert_rules!() + pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) + pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) + pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) + pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) + pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", PyIterable, PyIterable) + pyconvert_add_rule(pyconvert_rule_sequence, "collections.abc:Sequence", PyList, PyList) + pyconvert_add_rule(pyconvert_rule_set, "collections.abc:Set", PySet, PySet) + pyconvert_add_rule(pyconvert_rule_mapping, "collections.abc:Mapping", PyDict, PyDict) + pyconvert_add_rule(pyconvert_rule_io, "io:IOBase", PyIO, PyIO) + pyconvert_add_rule(pyconvert_rule_io, "_io:_IOBase", PyIO, PyIO) + pyconvert_add_rule(pyconvert_rule_pandasdataframe, "pandas.core.frame:DataFrame", PyPandasDataFrame, PyPandasDataFrame) + pyconvert_add_rule(pyconvert_rule_sequence, "pandas.core.arrays.base:ExtensionArray", PyList, PyList) + pyconvert_add_rule(pyconvert_rule_array, "", Array, Array) + pyconvert_add_rule(pyconvert_rule_array, "", Array, Array) + pyconvert_add_rule(pyconvert_rule_array, "", Array, Array) + pyconvert_add_rule(pyconvert_rule_array, "", Array, Array) + pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray, AbstractArray) + pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray, AbstractArray) + pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray, AbstractArray) + pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray, AbstractArray) end end From f42e34af854c13d492b2bea047728b7aa15299ce Mon Sep 17 00:00:00 2001 From: Christopher Rowley Date: Mon, 17 Nov 2025 10:01:15 +0000 Subject: [PATCH 09/11] Inline pyconvert registration --- src/Convert/ctypes.jl | 5 - src/Convert/numpy.jl | 5 - src/Convert/pandas.jl | 5 - src/Convert/pyconvert.jl | 46 ------ src/JlWrap/base.jl | 4 - src/PythonCall.jl | 333 +++++++++++++++++++++++++++++++++++++-- src/Wrap/Wrap.jl | 23 --- 7 files changed, 324 insertions(+), 97 deletions(-) diff --git a/src/Convert/ctypes.jl b/src/Convert/ctypes.jl index 89ee9975..88d513ff 100644 --- a/src/Convert/ctypes.jl +++ b/src/Convert/ctypes.jl @@ -72,8 +72,3 @@ function ctypes_rule_specs() return specs end -function register_ctypes_rules!() - for rule in ctypes_rule_specs() - pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) - end -end diff --git a/src/Convert/numpy.jl b/src/Convert/numpy.jl index b4e95059..09ed147b 100644 --- a/src/Convert/numpy.jl +++ b/src/Convert/numpy.jl @@ -183,8 +183,3 @@ function numpy_rule_specs() return specs end -function register_numpy_rules!() - for rule in numpy_rule_specs() - pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) - end -end diff --git a/src/Convert/pandas.jl b/src/Convert/pandas.jl index 5413fe6c..356949bc 100644 --- a/src/Convert/pandas.jl +++ b/src/Convert/pandas.jl @@ -13,8 +13,3 @@ function pandas_rule_specs() ] end -function register_pandas_rules!() - for rule in pandas_rule_specs() - pyconvert_add_rule(rule.func, rule.tname, rule.type, rule.scope) - end -end diff --git a/src/Convert/pyconvert.jl b/src/Convert/pyconvert.jl index 0d976c09..bedcaf4c 100644 --- a/src/Convert/pyconvert.jl +++ b/src/Convert/pyconvert.jl @@ -435,49 +435,3 @@ function init_pyconvert_extratypes() pyimport("collections.abc" => ("Iterable", "Sequence", "Set", "Mapping"))..., ) end - -function register_pyconvert_rules!() - pyconvert_add_rule(pyconvert_rule_none, "builtins:NoneType", Nothing, Any) - pyconvert_add_rule(pyconvert_rule_bool, "builtins:bool", Bool, Any) - pyconvert_add_rule(pyconvert_rule_float, "builtins:float", Float64, Any) - pyconvert_add_rule(pyconvert_rule_complex, "builtins:complex", Complex{Float64}, Any) - pyconvert_add_rule(pyconvert_rule_int, "numbers:Integral", Integer, Any) - pyconvert_add_rule(pyconvert_rule_str, "builtins:str", String, Any) - pyconvert_add_rule(pyconvert_rule_bytes, "builtins:bytes", Base.CodeUnits{UInt8,String}, Any) - pyconvert_add_rule(pyconvert_rule_range, "builtins:range", StepRange{<:Integer,<:Integer}, Any) - pyconvert_add_rule(pyconvert_rule_fraction, "numbers:Rational", Rational{<:Integer}, Any) - pyconvert_add_rule(pyconvert_rule_iterable, "builtins:tuple", NamedTuple, Any) - pyconvert_add_rule(pyconvert_rule_iterable, "builtins:tuple", Tuple, Any) - pyconvert_add_rule(pyconvert_rule_datetime, "datetime:datetime", DateTime, Any) - pyconvert_add_rule(pyconvert_rule_date, "datetime:date", Date, Any) - pyconvert_add_rule(pyconvert_rule_time, "datetime:time", Time, Any) - pyconvert_add_rule(pyconvert_rule_timedelta, "datetime:timedelta", Microsecond, Any) - pyconvert_add_rule(pyconvert_rule_exception, "builtins:BaseException", PyException, Any) - pyconvert_add_rule(pyconvert_rule_none, "builtins:NoneType", Missing, Missing) - pyconvert_add_rule(pyconvert_rule_bool, "builtins:bool", Number, Number) - pyconvert_add_rule(pyconvert_rule_float, "numbers:Real", Number, Number) - pyconvert_add_rule(pyconvert_rule_float, "builtins:float", Nothing, Nothing) - pyconvert_add_rule(pyconvert_rule_float, "builtins:float", Missing, Missing) - pyconvert_add_rule(pyconvert_rule_complex, "numbers:Complex", Number, Number) - pyconvert_add_rule(pyconvert_rule_int, "numbers:Integral", Number, Number) - pyconvert_add_rule(pyconvert_rule_str, "builtins:str", Symbol, Symbol) - pyconvert_add_rule(pyconvert_rule_str, "builtins:str", Char, Char) - pyconvert_add_rule(pyconvert_rule_bytes, "builtins:bytes", Vector{UInt8}, Vector{UInt8}) - pyconvert_add_rule(pyconvert_rule_range, "builtins:range", UnitRange{<:Integer}, UnitRange{<:Integer}) - pyconvert_add_rule(pyconvert_rule_fraction, "numbers:Rational", Number, Number) - pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", Vector, Vector) - pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", Tuple, Tuple) - pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", Pair, Pair) - pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", Set, Set) - pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Sequence", Vector, Vector) - pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Sequence", Tuple, Tuple) - pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Set", Set, Set) - pyconvert_add_rule(pyconvert_rule_mapping, "collections.abc:Mapping", Dict, Dict) - pyconvert_add_rule(pyconvert_rule_timedelta, "datetime:timedelta", Millisecond, Millisecond) - pyconvert_add_rule(pyconvert_rule_timedelta, "datetime:timedelta", Second, Second) - pyconvert_add_rule(pyconvert_rule_timedelta, "datetime:timedelta", Nanosecond, Nanosecond) -end - -function register_pyconvert_fallback_rules!() - pyconvert_add_rule(pyconvert_rule_object, "builtins:object", Py, Any) -end diff --git a/src/JlWrap/base.jl b/src/JlWrap/base.jl index d82581b1..fce60a12 100644 --- a/src/JlWrap/base.jl +++ b/src/JlWrap/base.jl @@ -25,10 +25,6 @@ function init_base() pyjuliacallmodule.JlBase = pyjlbasetype end -function register_jlwrap_rules!() - pyconvert_add_rule(pyconvert_rule_jlvalue, "juliacall:JlBase", Any, Any) -end - pyconvert_rule_jlvalue(::Type{T}, x::Py) where {T} = pyconvert_tryconvert(T, _pyjl_getvalue(x)) diff --git a/src/PythonCall.jl b/src/PythonCall.jl index eb632c2c..770f7dcd 100644 --- a/src/PythonCall.jl +++ b/src/PythonCall.jl @@ -2,6 +2,8 @@ module PythonCall const ROOT_DIR = dirname(@__DIR__) +using Dates: Date, DateTime, Microsecond, Millisecond, Nanosecond, Second, Time + include("API/API.jl") include("Utils/Utils.jl") include("NumpyDates/NumpyDates.jl") @@ -32,15 +34,328 @@ for k in [ end function __init__() - Convert.init_pyconvert_extratypes() - - Convert.register_pyconvert_rules!() - Convert.register_ctypes_rules!() - Convert.register_numpy_rules!() - Convert.register_pandas_rules!() - Wrap.register_wrap_pyconvert_rules!() - JlWrap.register_jlwrap_rules!() - Convert.register_pyconvert_fallback_rules!() + # Core pyconvert rules + pyconvert_add_rule(Convert.pyconvert_rule_none, "builtins:NoneType", Nothing, Any) + pyconvert_add_rule(Convert.pyconvert_rule_bool, "builtins:bool", Bool, Any) + pyconvert_add_rule(Convert.pyconvert_rule_float, "builtins:float", Float64, Any) + pyconvert_add_rule(Convert.pyconvert_rule_complex, "builtins:complex", Complex{Float64}, Any) + pyconvert_add_rule(Convert.pyconvert_rule_int, "numbers:Integral", Integer, Any) + pyconvert_add_rule(Convert.pyconvert_rule_str, "builtins:str", String, Any) + pyconvert_add_rule(Convert.pyconvert_rule_bytes, "builtins:bytes", Base.CodeUnits{UInt8,String}, Any) + pyconvert_add_rule( + Convert.pyconvert_rule_range, + "builtins:range", + StepRange{<:Integer,<:Integer}, + Any, + ) + pyconvert_add_rule(Convert.pyconvert_rule_fraction, "numbers:Rational", Rational{<:Integer}, Any) + pyconvert_add_rule(Convert.pyconvert_rule_iterable, "builtins:tuple", NamedTuple, Any) + pyconvert_add_rule(Convert.pyconvert_rule_iterable, "builtins:tuple", Tuple, Any) + pyconvert_add_rule(Convert.pyconvert_rule_datetime, "datetime:datetime", DateTime, Any) + pyconvert_add_rule(Convert.pyconvert_rule_date, "datetime:date", Date, Any) + pyconvert_add_rule(Convert.pyconvert_rule_time, "datetime:time", Time, Any) + pyconvert_add_rule(Convert.pyconvert_rule_timedelta, "datetime:timedelta", Microsecond, Any) + pyconvert_add_rule(Convert.pyconvert_rule_exception, "builtins:BaseException", PyException, Any) + pyconvert_add_rule(Convert.pyconvert_rule_none, "builtins:NoneType", Missing, Missing) + pyconvert_add_rule(Convert.pyconvert_rule_bool, "builtins:bool", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_float, "numbers:Real", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_float, "builtins:float", Nothing, Nothing) + pyconvert_add_rule(Convert.pyconvert_rule_float, "builtins:float", Missing, Missing) + pyconvert_add_rule(Convert.pyconvert_rule_complex, "numbers:Complex", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_int, "numbers:Integral", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_str, "builtins:str", Symbol, Symbol) + pyconvert_add_rule(Convert.pyconvert_rule_str, "builtins:str", Char, Char) + pyconvert_add_rule(Convert.pyconvert_rule_bytes, "builtins:bytes", Vector{UInt8}, Vector{UInt8}) + pyconvert_add_rule( + Convert.pyconvert_rule_range, + "builtins:range", + UnitRange{<:Integer}, + UnitRange{<:Integer}, + ) + pyconvert_add_rule(Convert.pyconvert_rule_fraction, "numbers:Rational", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_iterable, "collections.abc:Iterable", Vector, Vector) + pyconvert_add_rule(Convert.pyconvert_rule_iterable, "collections.abc:Iterable", Tuple, Tuple) + pyconvert_add_rule(Convert.pyconvert_rule_iterable, "collections.abc:Iterable", Pair, Pair) + pyconvert_add_rule(Convert.pyconvert_rule_iterable, "collections.abc:Iterable", Set, Set) + pyconvert_add_rule(Convert.pyconvert_rule_iterable, "collections.abc:Sequence", Vector, Vector) + pyconvert_add_rule(Convert.pyconvert_rule_iterable, "collections.abc:Sequence", Tuple, Tuple) + pyconvert_add_rule(Convert.pyconvert_rule_iterable, "collections.abc:Set", Set, Set) + pyconvert_add_rule(Convert.pyconvert_rule_mapping, "collections.abc:Mapping", Dict, Dict) + pyconvert_add_rule(Convert.pyconvert_rule_timedelta, "datetime:timedelta", Millisecond, Millisecond) + pyconvert_add_rule(Convert.pyconvert_rule_timedelta, "datetime:timedelta", Second, Second) + pyconvert_add_rule(Convert.pyconvert_rule_timedelta, "datetime:timedelta", Nanosecond, Nanosecond) + + # ctypes rules + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,true}(), "ctypes:c_char", Cchar, Cchar) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,true}(), "ctypes:c_char", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,false}(), "ctypes:c_char", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,false}(), "ctypes:c_char", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,false}(), "ctypes:c_char", Number, Number) + pyconvert_add_rule( + Convert.pyconvert_rule_ctypessimplevalue{Cwchar_t,true}(), + "ctypes:c_wchar", + Cwchar_t, + Cwchar_t, + ) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cwchar_t,true}(), "ctypes:c_wchar", Int, Int) + pyconvert_add_rule( + Convert.pyconvert_rule_ctypessimplevalue{Cwchar_t,false}(), + "ctypes:c_wchar", + Integer, + Integer, + ) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cwchar_t,false}(), "ctypes:c_wchar", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cwchar_t,false}(), "ctypes:c_wchar", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,true}(), "ctypes:c_byte", Cchar, Cchar) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,true}(), "ctypes:c_byte", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,false}(), "ctypes:c_byte", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,false}(), "ctypes:c_byte", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,false}(), "ctypes:c_byte", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuchar,true}(), "ctypes:c_ubyte", Cuchar, Cuchar) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuchar,true}(), "ctypes:c_ubyte", UInt, UInt) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuchar,true}(), "ctypes:c_ubyte", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuchar,false}(), "ctypes:c_ubyte", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuchar,false}(), "ctypes:c_ubyte", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuchar,false}(), "ctypes:c_ubyte", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cshort,true}(), "ctypes:c_short", Cshort, Cshort) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cshort,true}(), "ctypes:c_short", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cshort,false}(), "ctypes:c_short", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cshort,false}(), "ctypes:c_short", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cshort,false}(), "ctypes:c_short", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cushort,true}(), "ctypes:c_ushort", Cushort, Cushort) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cushort,true}(), "ctypes:c_ushort", UInt, UInt) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cushort,true}(), "ctypes:c_ushort", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cushort,false}(), "ctypes:c_ushort", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cushort,false}(), "ctypes:c_ushort", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cushort,false}(), "ctypes:c_ushort", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cint,true}(), "ctypes:c_int", Cint, Cint) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cint,true}(), "ctypes:c_int", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cint,false}(), "ctypes:c_int", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cint,false}(), "ctypes:c_int", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cint,false}(), "ctypes:c_int", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuint,true}(), "ctypes:c_uint", Cuint, Cuint) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuint,true}(), "ctypes:c_uint", UInt, UInt) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuint,true}(), "ctypes:c_uint", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuint,false}(), "ctypes:c_uint", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuint,false}(), "ctypes:c_uint", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuint,false}(), "ctypes:c_uint", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clong,true}(), "ctypes:c_long", Clong, Clong) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clong,true}(), "ctypes:c_long", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clong,false}(), "ctypes:c_long", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clong,false}(), "ctypes:c_long", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clong,false}(), "ctypes:c_long", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culong,true}(), "ctypes:c_ulong", Culong, Culong) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culong,true}(), "ctypes:c_ulong", UInt, UInt) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culong,false}(), "ctypes:c_ulong", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culong,false}(), "ctypes:c_ulong", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culong,false}(), "ctypes:c_ulong", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culong,false}(), "ctypes:c_ulong", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clonglong,true}(), "ctypes:c_longlong", Clonglong, Clonglong) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clonglong,true}(), "ctypes:c_longlong", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clonglong,false}(), "ctypes:c_longlong", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clonglong,false}(), "ctypes:c_longlong", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clonglong,false}(), "ctypes:c_longlong", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culonglong,true}(), "ctypes:c_ulonglong", Culonglong, Culonglong) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culonglong,true}(), "ctypes:c_ulonglong", UInt, UInt) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culonglong,false}(), "ctypes:c_ulonglong", Int, Int) + pyconvert_add_rule( + Convert.pyconvert_rule_ctypessimplevalue{Culonglong,false}(), + "ctypes:c_ulonglong", + Integer, + Integer, + ) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culonglong,false}(), "ctypes:c_ulonglong", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culonglong,false}(), "ctypes:c_ulonglong", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Csize_t,true}(), "ctypes:c_size_t", Csize_t, Csize_t) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Csize_t,true}(), "ctypes:c_size_t", UInt, UInt) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Csize_t,false}(), "ctypes:c_size_t", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Csize_t,false}(), "ctypes:c_size_t", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Csize_t,false}(), "ctypes:c_size_t", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Csize_t,false}(), "ctypes:c_size_t", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cssize_t,true}(), "ctypes:c_ssize_t", Cssize_t, Cssize_t) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cssize_t,true}(), "ctypes:c_ssize_t", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cssize_t,false}(), "ctypes:c_ssize_t", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cssize_t,false}(), "ctypes:c_ssize_t", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cssize_t,false}(), "ctypes:c_ssize_t", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cfloat,true}(), "ctypes:c_float", Cfloat, Cfloat) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cfloat,true}(), "ctypes:c_float", Float64, Float64) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cfloat,false}(), "ctypes:c_float", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cfloat,false}(), "ctypes:c_float", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cdouble,true}(), "ctypes:c_double", Cdouble, Cdouble) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cdouble,true}(), "ctypes:c_double", Float64, Float64) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cdouble,false}(), "ctypes:c_double", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cdouble,false}(), "ctypes:c_double", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cchar},true}(), "ctypes:c_char_p", Cstring, Cstring) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cchar},true}(), "ctypes:c_char_p", Ptr{Cchar}, Ptr{Cchar}) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cchar},true}(), "ctypes:c_char_p", Ptr, Ptr) + pyconvert_add_rule( + Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cwchar_t},true}(), + "ctypes:c_wchar_p", + Cwstring, + Cwstring, + ) + pyconvert_add_rule( + Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cwchar_t},true}(), + "ctypes:c_wchar_p", + Ptr{Cwchar_t}, + Ptr{Cwchar_t}, + ) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cwchar_t},true}(), "ctypes:c_wchar_p", Ptr, Ptr) + pyconvert_add_rule( + Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cvoid},true}(), + "ctypes:c_void_p", + Ptr{Cvoid}, + Ptr{Cvoid}, + ) + pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cvoid},true}(), "ctypes:c_void_p", Ptr, Ptr) + + # numpy rules + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Bool,true}(), "numpy:bool_", Bool, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Bool,true}(), "numpy:bool_", UInt, UInt) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Bool,true}(), "numpy:bool_", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Bool,false}(), "numpy:bool_", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Bool,false}(), "numpy:bool_", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Bool,false}(), "numpy:bool_", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int8,true}(), "numpy:int8", Int8, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int8,true}(), "numpy:int8", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int8,false}(), "numpy:int8", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int8,false}(), "numpy:int8", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int8,false}(), "numpy:int8", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int16,true}(), "numpy:int16", Int16, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int16,true}(), "numpy:int16", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int16,false}(), "numpy:int16", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int16,false}(), "numpy:int16", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int16,false}(), "numpy:int16", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int32,true}(), "numpy:int32", Int32, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int32,true}(), "numpy:int32", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int32,false}(), "numpy:int32", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int32,false}(), "numpy:int32", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int32,false}(), "numpy:int32", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int64,true}(), "numpy:int64", Int64, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int64,true}(), "numpy:int64", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int64,false}(), "numpy:int64", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int64,false}(), "numpy:int64", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int64,false}(), "numpy:int64", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt8,true}(), "numpy:uint8", UInt8, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt8,true}(), "numpy:uint8", UInt, UInt) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt8,true}(), "numpy:uint8", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt8,false}(), "numpy:uint8", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt8,false}(), "numpy:uint8", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt8,false}(), "numpy:uint8", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt16,true}(), "numpy:uint16", UInt16, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt16,true}(), "numpy:uint16", UInt, UInt) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt16,true}(), "numpy:uint16", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt16,false}(), "numpy:uint16", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt16,false}(), "numpy:uint16", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt16,false}(), "numpy:uint16", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt32,true}(), "numpy:uint32", UInt32, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt32,true}(), "numpy:uint32", UInt, UInt) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt32,true}(), "numpy:uint32", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt32,false}(), "numpy:uint32", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt32,false}(), "numpy:uint32", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt32,false}(), "numpy:uint32", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt64,true}(), "numpy:uint64", UInt64, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt64,true}(), "numpy:uint64", UInt, UInt) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt64,false}(), "numpy:uint64", Int, Int) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt64,false}(), "numpy:uint64", Integer, Integer) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt64,false}(), "numpy:uint64", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt64,false}(), "numpy:uint64", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float16,true}(), "numpy:float16", Float16, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float16,true}(), "numpy:float16", Float64, Float64) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float16,false}(), "numpy:float16", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float16,false}(), "numpy:float16", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float32,true}(), "numpy:float32", Float32, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float32,true}(), "numpy:float32", Float64, Float64) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float32,false}(), "numpy:float32", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float32,false}(), "numpy:float32", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float64,true}(), "numpy:float64", Float64, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float64,true}(), "numpy:float64", Float64, Float64) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float64,false}(), "numpy:float64", Real, Real) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float64,false}(), "numpy:float64", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF16,true}(), "numpy:complex32", ComplexF16, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF16,true}(), "numpy:complex32", ComplexF64, ComplexF64) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF16,false}(), "numpy:complex32", Complex, Complex) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF16,false}(), "numpy:complex32", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF32,true}(), "numpy:complex64", ComplexF32, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF32,true}(), "numpy:complex64", ComplexF64, ComplexF64) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF32,false}(), "numpy:complex64", Complex, Complex) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF32,false}(), "numpy:complex64", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF64,true}(), "numpy:complex128", ComplexF64, Any) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF64,true}(), "numpy:complex128", ComplexF64, ComplexF64) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF64,false}(), "numpy:complex128", Complex, Complex) + pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF64,false}(), "numpy:complex128", Number, Number) + pyconvert_add_rule(Convert.pyconvert_rule_datetime64, "numpy:datetime64", NumpyDates.DateTime64, Any) + pyconvert_add_rule(Convert.pyconvert_rule_datetime64, "numpy:datetime64", NumpyDates.InlineDateTime64, NumpyDates.InlineDateTime64) + pyconvert_add_rule( + Convert.pyconvert_rule_datetime64, + "numpy:datetime64", + NumpyDates.DatesInstant, + NumpyDates.DatesInstant, + ) + pyconvert_add_rule(Convert.pyconvert_rule_datetime64, "numpy:datetime64", Missing, Missing) + pyconvert_add_rule(Convert.pyconvert_rule_datetime64, "numpy:datetime64", Nothing, Nothing) + pyconvert_add_rule(Convert.pyconvert_rule_timedelta64, "numpy:timedelta64", NumpyDates.TimeDelta64, Any) + pyconvert_add_rule( + Convert.pyconvert_rule_timedelta64, + "numpy:timedelta64", + NumpyDates.InlineTimeDelta64, + NumpyDates.InlineTimeDelta64, + ) + pyconvert_add_rule( + Convert.pyconvert_rule_timedelta64, + "numpy:timedelta64", + NumpyDates.DatesPeriod, + NumpyDates.DatesPeriod, + ) + pyconvert_add_rule(Convert.pyconvert_rule_timedelta64, "numpy:timedelta64", Missing, Missing) + pyconvert_add_rule(Convert.pyconvert_rule_timedelta64, "numpy:timedelta64", Nothing, Nothing) + + # pandas rules + pyconvert_add_rule( + Convert.pyconvert_rule_pandas_na, + "pandas._libs.missing:NAType", + Missing, + Any, + ) + pyconvert_add_rule(Convert.pyconvert_rule_pandas_na, "pandas._libs.missing:NAType", Nothing, Nothing) + + # wrapper rules + pyconvert_add_rule(Wrap.pyconvert_rule_array_nocopy, "", Wrap.PyArray, Any) + pyconvert_add_rule(Wrap.pyconvert_rule_array_nocopy, "", Wrap.PyArray, Any) + pyconvert_add_rule(Wrap.pyconvert_rule_array_nocopy, "", Wrap.PyArray, Any) + pyconvert_add_rule(Wrap.pyconvert_rule_array_nocopy, "", Wrap.PyArray, Any) + pyconvert_add_rule(Wrap.pyconvert_rule_iterable, "collections.abc:Iterable", Wrap.PyIterable, Wrap.PyIterable) + pyconvert_add_rule(Wrap.pyconvert_rule_sequence, "collections.abc:Sequence", Wrap.PyList, Wrap.PyList) + pyconvert_add_rule(Wrap.pyconvert_rule_set, "collections.abc:Set", Wrap.PySet, Wrap.PySet) + pyconvert_add_rule(Wrap.pyconvert_rule_mapping, "collections.abc:Mapping", Wrap.PyDict, Wrap.PyDict) + pyconvert_add_rule(Wrap.pyconvert_rule_io, "io:IOBase", Wrap.PyIO, Wrap.PyIO) + pyconvert_add_rule(Wrap.pyconvert_rule_io, "_io:_IOBase", Wrap.PyIO, Wrap.PyIO) + pyconvert_add_rule( + Wrap.pyconvert_rule_pandasdataframe, + "pandas.core.frame:DataFrame", + Wrap.PyPandasDataFrame, + Wrap.PyPandasDataFrame, + ) + pyconvert_add_rule( + Wrap.pyconvert_rule_sequence, + "pandas.core.arrays.base:ExtensionArray", + Wrap.PyList, + Wrap.PyList, + ) + pyconvert_add_rule(Wrap.pyconvert_rule_array, "", Array, Array) + pyconvert_add_rule(Wrap.pyconvert_rule_array, "", Array, Array) + pyconvert_add_rule(Wrap.pyconvert_rule_array, "", Array, Array) + pyconvert_add_rule(Wrap.pyconvert_rule_array, "", Array, Array) + pyconvert_add_rule(Wrap.pyconvert_rule_array, "", AbstractArray, AbstractArray) + pyconvert_add_rule(Wrap.pyconvert_rule_array, "", AbstractArray, AbstractArray) + pyconvert_add_rule(Wrap.pyconvert_rule_array, "", AbstractArray, AbstractArray) + pyconvert_add_rule(Wrap.pyconvert_rule_array, "", AbstractArray, AbstractArray) + + # JlWrap rules + pyconvert_add_rule(JlWrap.pyconvert_rule_jlvalue, "juliacall:JlBase", Any, Any) + + # Fallback + pyconvert_add_rule(Convert.pyconvert_rule_object, "builtins:object", Py, Any) end end diff --git a/src/Wrap/Wrap.jl b/src/Wrap/Wrap.jl index e9a667c8..e5032d2d 100644 --- a/src/Wrap/Wrap.jl +++ b/src/Wrap/Wrap.jl @@ -31,27 +31,4 @@ include("PyIO.jl") include("PyTable.jl") include("PyPandasDataFrame.jl") -function register_wrap_pyconvert_rules!() - pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) - pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) - pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) - pyconvert_add_rule(pyconvert_rule_array_nocopy, "", PyArray, Any) - pyconvert_add_rule(pyconvert_rule_iterable, "collections.abc:Iterable", PyIterable, PyIterable) - pyconvert_add_rule(pyconvert_rule_sequence, "collections.abc:Sequence", PyList, PyList) - pyconvert_add_rule(pyconvert_rule_set, "collections.abc:Set", PySet, PySet) - pyconvert_add_rule(pyconvert_rule_mapping, "collections.abc:Mapping", PyDict, PyDict) - pyconvert_add_rule(pyconvert_rule_io, "io:IOBase", PyIO, PyIO) - pyconvert_add_rule(pyconvert_rule_io, "_io:_IOBase", PyIO, PyIO) - pyconvert_add_rule(pyconvert_rule_pandasdataframe, "pandas.core.frame:DataFrame", PyPandasDataFrame, PyPandasDataFrame) - pyconvert_add_rule(pyconvert_rule_sequence, "pandas.core.arrays.base:ExtensionArray", PyList, PyList) - pyconvert_add_rule(pyconvert_rule_array, "", Array, Array) - pyconvert_add_rule(pyconvert_rule_array, "", Array, Array) - pyconvert_add_rule(pyconvert_rule_array, "", Array, Array) - pyconvert_add_rule(pyconvert_rule_array, "", Array, Array) - pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray, AbstractArray) - pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray, AbstractArray) - pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray, AbstractArray) - pyconvert_add_rule(pyconvert_rule_array, "", AbstractArray, AbstractArray) -end - end From 5eca22e8621e89aed2e69e60ef16bed939d56329 Mon Sep 17 00:00:00 2001 From: Christopher Rowley Date: Tue, 18 Nov 2025 19:07:55 +0000 Subject: [PATCH 10/11] Refine pyconvert registration and docs --- CHANGELOG.md | 2 + docs/src/conversion-to-julia.md | 116 ++++++++------- src/Convert/ctypes.jl | 38 ----- src/Convert/numpy.jl | 86 ----------- src/Convert/pandas.jl | 12 -- src/Convert/pyconvert.jl | 1 - src/PythonCall.jl | 247 +++++++------------------------- 7 files changed, 116 insertions(+), 386 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 622a2be9..11a5591e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,8 @@ * Changes to core functionality: * Comparisons like `==(::Py, ::Py)`, `<(::Py, ::Number)`, `isless(::Number, ::Py)` now return `Bool` instead of `Py`. * `pyconvert` rules are now scoped by target type instead of prioritized; rules are ordered by Python type specificity and creation order. +### Conversion +* `pyconvert_add_rule` now has the signature `pyconvert_add_rule(func::Function, t::String, ::Type{T}, ::Type{S}=T)`; the optional scope `S` controls when a rule is considered during conversion. * Changes to `PythonCall.GC` (now more like `Base.GC`): * `enable(true)` replaces `enable()`. * `enable(false)` replaces `disable()`. diff --git a/docs/src/conversion-to-julia.md b/docs/src/conversion-to-julia.md index 7d898f27..8168ba66 100644 --- a/docs/src/conversion-to-julia.md +++ b/docs/src/conversion-to-julia.md @@ -2,63 +2,75 @@ ## [Conversion Rules](@id py2jl-conversion) -The following table specifies the conversion rules used whenever converting a Python object to a Julia object. If the initial Python type matches the "From" column and the desired type `T` intersects with the "To" column, then that conversion is attempted. Rules are ordered by Python type specificity (strict subclassing only) and then by creation order. A rule only applies when the requested target type is a subtype of its scope; unless otherwise noted, the scope matches the type in the "To" column, so those rules apply when you explicitly request that type. +The following table specifies the conversion rules used whenever converting a Python object to a Julia object. If the initial Python type matches the "From" column and the desired type `T` intersects with the "To" column, then that conversion is attempted. Rules are ordered by Python type specificity (strict subclassing only) and then by creation order. A rule only applies when the requested target type is a subtype of its scope; the "Scope" column lists the type that must match. From Julia, one can convert Python objects to a desired type using `pyconvert(T, x)` for example. From Python, the arguments to a Julia function will be converted according to these rules with `T=Any`. -| From | To | -| :----------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------- | -| **Default-scope rules (apply even when converting to `Any`).** | - | -| `juliacall.Jl` | `Any` | -| Objects satisfying the buffer or array interface (inc. `bytes`, `bytearray`, `array.array`, `numpy.ndarray`) | `PyArray` | -| **Default conversions for common types.** | -| `None` | `Nothing` | -| `bool` | `Bool` | -| `numbers.Integral` (inc. `int`) | `Integer` (prefers `Int`, or `BigInt` on overflow) | -| `float` | `Float64` | -| `complex` | `Complex{Float64}` | -| `range` | `StepRange` | -| `str` | `String` | -| `tuple` | `Tuple` | -| `collections.abc.Mapping` (inc. `dict`) | `PyDict` | -| `collections.abc.Sequence` (inc. `list`) | `PyList` | -| `collections.abc.Set` (inc. `set`, `frozenset`) | `PySet` | -| `io.IOBase` (includes open files) | `PyIO` | -| `BaseException` | `PyException` | -| `datetime.date`/`datetime.time`/`datetime.datetime` | `Date`/`Time`/`DateTime` | -| `datetime.timedelta` | `Microsecond` (or `Millisecond` or `Second` on overflow) | -| `numpy.intXX`/`numpy.uintXX`/`numpy.floatXX` | `IntXX`/`UIntXX`/`FloatXX` | -| `numpy.datetime64` | `NumpyDates.DateTime64` | -| `numpy.timedelta64` | `NumpyDates.TimeDelta64` | -| **Additional conversions requiring matching target scope.** | - | -| `None` | `Missing` (scope `Missing`) | -| `bytes` | `Vector{UInt8}`, `Vector{Int8}`, `String` | -| `str` | `String`, `Symbol`, `Char`, `Vector{UInt8}`, `Vector{Int8}` | -| `range` | `UnitRange` | -| `collections.abc.Mapping` | `Dict` | -| `collections.abc.Iterable` | `Vector`, `Set`, `Tuple`, `NamedTuple`, `Pair` | -| `datetime.timedelta` | `Dates.CompoundPeriod` | -| `numbers.Integral` | `Integer`, `Rational`, `Real`, `Number` | -| `numbers.Real` | `AbstractFloat`, `Number`, `Missing`/`Nothing` (if NaN) | -| `numbers.Complex` | `Complex`, `Number` | -| `ctypes.c_int` and other integers | `Integer`, `Rational`, `Real`, `Number` | -| `ctypes.c_float`/`ctypes.c_double` | `Cfloat`/`Cdouble`, `AbstractFloat`, `Real`, `Number` | -| `ctypes.c_voidp` | `Ptr{Cvoid}`, `Ptr` | -| `ctypes.c_char_p` | `Cstring`, `Ptr{Cchar}`, `Ptr` | -| `ctypes.c_wchar_p` | `Cwstring`, `Ptr{Cwchar}`, `Ptr` | -| `numpy.bool_`/`numpy.intXX`/`numpy.uintXX`/`numpy.floatXX` | `Bool`, `Integer`, `Rational`, `Real`, `Number` | -| `numpy.datetime64` | `NumpyDates.InlineDateTime64`, `Dates.DateTime` | -| `numpy.timedelta64` | `NumpyDates.InlineTimeDelta64`, `Dates.Period` | -| **Fallback conversion.** | - | -| Anything | `Py` | -| **Explicit wrapper conversions (require excluding `Py`).** | - | -| Anything | `PyRef` | +| From | To | Scope | +| :----------------------------------------------------------------------------------------------------------- | :--------------------------------------------- | :---- | +| **Default-scope rules (apply even when converting to `Any`).** | | | +| `juliacall.Jl` | `Any` | `Any` | +| Objects satisfying the buffer or array interface (inc. `bytes`, `bytearray`, `array.array`, `numpy.ndarray`) | `PyArray` | `Any` | +| `None` | `Nothing` | `Any` | +| `bool` | `Bool` | `Any` | +| `numbers.Integral` (inc. `int`) | `Integer` (prefers `Int`, or `BigInt` on overflow) | `Any` | +| `numbers.Rational` | `Rational{<:Integer}` | `Any` | +| `float` | `Float64` | `Any` | +| `complex` | `Complex{Float64}` | `Any` | +| `range` | `StepRange{<:Integer,<:Integer}` | `Any` | +| `str` | `String` | `Any` | +| `bytes` | `Base.CodeUnits{UInt8,String}` | `Any` | +| `tuple` | `NamedTuple` | `Any` | +| `tuple` | `Tuple` | `Any` | +| `datetime.datetime` | `DateTime` | `Any` | +| `datetime.date` | `Date` | `Any` | +| `datetime.time` | `Time` | `Any` | +| `datetime.timedelta` | `Microsecond` | `Any` | +| `numpy.bool_`/`numpy.intXX`/`numpy.uintXX`/`numpy.floatXX`/`numpy.complexXX` | matching Julia scalar | `Any` | +| `numpy.datetime64` | `NumpyDates.DateTime64` | `Any` | +| `numpy.timedelta64` | `NumpyDates.TimeDelta64` | `Any` | +| `pandas._libs.missing.NAType` | `Missing` | `Any` | +| `juliacall.JlBase` | `Any` | `Any` | +| `builtins.object` | `Py` | `Any` | +| **Scoped conversions (only when the requested target matches the scope).** | | | +| `None` | `Missing` | `Missing` | +| `bool` | `Number` | `Number` | +| `float` | `Number` | `Number` | +| `float` (NaN) | `Nothing` / `Missing` | `Nothing` / `Missing` | +| `complex` | `Number` | `Number` | +| `numbers.Integral` | `Number` | `Number` | +| `numbers.Rational` | `Number` | `Number` | +| `str` | `Symbol` | `Symbol` | +| `str` | `Char` | `Char` | +| `bytes` | `Vector{UInt8}` | `Vector{UInt8}` | +| `range` | `UnitRange{<:Integer}` | `UnitRange{<:Integer}` | +| `collections.abc.Iterable` | `Vector` / `Tuple` / `Set` / `NamedTuple` / `Pair` | respective targets | +| `collections.abc.Sequence` | `Vector` / `Tuple` | respective targets | +| `collections.abc.Set` | `Set` | `Set` | +| `collections.abc.Mapping` | `Dict` | `Dict` | +| `datetime.timedelta` | `Millisecond` / `Second` / `Nanosecond` | same as target | +| `numpy.datetime64` | `NumpyDates.InlineDateTime64` | `NumpyDates.InlineDateTime64` | +| `numpy.datetime64` | `NumpyDates.DatesInstant` | `NumpyDates.DatesInstant` | +| `numpy.datetime64` | `Missing` / `Nothing` | same as target | +| `numpy.timedelta64` | `NumpyDates.InlineTimeDelta64` | `NumpyDates.InlineTimeDelta64` | +| `numpy.timedelta64` | `NumpyDates.DatesPeriod` | `NumpyDates.DatesPeriod` | +| `numpy.timedelta64` | `Missing` / `Nothing` | same as target | +| NumPy scalars (`numpy.bool_`, `numpy.intXX`, `numpy.uintXX`, `numpy.floatXX`, `numpy.complexXX`) | `Int` / `UInt` / `Integer` / `Real` / `Complex{Float64}` / `Complex` / `Number` | scope matches target | +| ctypes simple values (`c_int`, `c_double`, `c_void_p`, `c_char_p`, etc.) | matching C type; widening numeric targets (`Int` / `UInt` / `Integer` / `Real` / `Number`); pointers (`Ptr`, `Cstring`, `Cwstring`) | scope matches target | +| `pandas._libs.missing.NAType` | `Nothing` | `Nothing` | +| **Wrapper conversions (opt-in scopes).** | | | +| `collections.abc.Iterable` | `PyIterable` | `PyIterable` | +| `collections.abc.Sequence` | `PyList` | `PyList` | +| `collections.abc.Set` | `PySet` | `PySet` | +| `collections.abc.Mapping` | `PyDict` | `PyDict` | +| `io.IOBase` / `_io._IOBase` | `PyIO` | `PyIO` | +| `pandas.core.frame.DataFrame` | `PyPandasDataFrame` | `PyPandasDataFrame` | +| `pandas.core.arrays.base.ExtensionArray` | `PyList` | `PyList` | +| Objects satisfying the buffer or array interface | `Array` / `AbstractArray` | same as target | +| **Explicit wrapper opt-out.** | | | +| Anything | `PyRef` | `PyRef` | See [here](@ref python-wrappers) for an explanation of the `Py*` wrapper types (`PyList`, `PyIO`, etc). diff --git a/src/Convert/ctypes.jl b/src/Convert/ctypes.jl index 88d513ff..efbce51e 100644 --- a/src/Convert/ctypes.jl +++ b/src/Convert/ctypes.jl @@ -34,41 +34,3 @@ const CTYPES_SIMPLE_TYPES = [ ("void_p", Ptr{Cvoid}), ] -function ctypes_rule_specs() - specs = PyConvertRuleSpec[] - for (t, T) in CTYPES_SIMPLE_TYPES - isptr = endswith(t, "_p") - isreal = !isptr - isnumber = isreal - isfloat = t in ("float", "double") - isint = isreal && !isfloat - isuint = isint && (startswith(t, "u") || t == "size_t") - - name = "ctypes:c_$t" - rule = pyconvert_rule_ctypessimplevalue{T,false}() - saferule = pyconvert_rule_ctypessimplevalue{T,true}() - - t == "char_p" && push!(specs, (func = saferule, tname = name, type = Cstring, scope = Cstring)) - t == "wchar_p" && push!(specs, (func = saferule, tname = name, type = Cwstring, scope = Cwstring)) - push!(specs, (func = saferule, tname = name, type = T, scope = T)) - isuint && push!( - specs, - (func = sizeof(T) ≤ sizeof(UInt) ? saferule : rule, tname = name, type = UInt, scope = UInt), - ) - isuint && push!( - specs, - (func = sizeof(T) < sizeof(Int) ? saferule : rule, tname = name, type = Int, scope = Int), - ) - isint && !isuint && push!( - specs, - (func = sizeof(T) ≤ sizeof(Int) ? saferule : rule, tname = name, type = Int, scope = Int), - ) - isint && push!(specs, (func = rule, tname = name, type = Integer, scope = Integer)) - isfloat && push!(specs, (func = saferule, tname = name, type = Float64, scope = Float64)) - isreal && push!(specs, (func = rule, tname = name, type = Real, scope = Real)) - isnumber && push!(specs, (func = rule, tname = name, type = Number, scope = Number)) - isptr && push!(specs, (func = saferule, tname = name, type = Ptr, scope = Ptr)) - end - return specs -end - diff --git a/src/Convert/numpy.jl b/src/Convert/numpy.jl index 09ed147b..276d0849 100644 --- a/src/Convert/numpy.jl +++ b/src/Convert/numpy.jl @@ -97,89 +97,3 @@ const NUMPY_SIMPLE_TYPES = [ ("complex128", ComplexF64), ] -function numpy_rule_specs() - specs = PyConvertRuleSpec[] - # simple numeric scalar types - for (t, T) in NUMPY_SIMPLE_TYPES - isbool = occursin("bool", t) - isint = occursin("int", t) || isbool - isuint = occursin("uint", t) || isbool - isfloat = occursin("float", t) - iscomplex = occursin("complex", t) - isreal = isint || isfloat - isnumber = isreal || iscomplex - - name = "numpy:$t" - rule = pyconvert_rule_numpysimplevalue{T,false}() - saferule = pyconvert_rule_numpysimplevalue{T,true}() - - push!(specs, (func = saferule, tname = name, type = T, scope = Any)) - isuint && push!( - specs, - ( - func = sizeof(T) ≤ sizeof(UInt) ? saferule : rule, - tname = name, - type = UInt, - scope = UInt, - ), - ) - isuint && push!( - specs, - ( - func = sizeof(T) < sizeof(Int) ? saferule : rule, - tname = name, - type = Int, - scope = Int, - ), - ) - isint && !isuint && push!( - specs, - (func = sizeof(T) ≤ sizeof(Int) ? saferule : rule, tname = name, type = Int, scope = Int), - ) - isint && push!(specs, (func = rule, tname = name, type = Integer, scope = Integer)) - isfloat && push!(specs, (func = saferule, tname = name, type = Float64, scope = Float64)) - isreal && push!(specs, (func = rule, tname = name, type = Real, scope = Real)) - iscomplex && push!(specs, (func = saferule, tname = name, type = ComplexF64, scope = ComplexF64)) - iscomplex && push!(specs, (func = rule, tname = name, type = Complex, scope = Complex)) - isnumber && push!(specs, (func = rule, tname = name, type = Number, scope = Number)) - end - - # datetime64 - push!( - specs, - (func = pyconvert_rule_datetime64, tname = "numpy:datetime64", type = DateTime64, scope = Any), - ) - push!(specs, (func = pyconvert_rule_datetime64, tname = "numpy:datetime64", type = InlineDateTime64, scope = InlineDateTime64)) - push!( - specs, - ( - func = pyconvert_rule_datetime64, - tname = "numpy:datetime64", - type = NumpyDates.DatesInstant, - scope = NumpyDates.DatesInstant, - ), - ) - push!(specs, (func = pyconvert_rule_datetime64, tname = "numpy:datetime64", type = Missing, scope = Missing)) - push!(specs, (func = pyconvert_rule_datetime64, tname = "numpy:datetime64", type = Nothing, scope = Nothing)) - - # timedelta64 - push!( - specs, - (func = pyconvert_rule_timedelta64, tname = "numpy:timedelta64", type = TimeDelta64, scope = Any), - ) - push!(specs, (func = pyconvert_rule_timedelta64, tname = "numpy:timedelta64", type = InlineTimeDelta64, scope = InlineTimeDelta64)) - push!( - specs, - ( - func = pyconvert_rule_timedelta64, - tname = "numpy:timedelta64", - type = NumpyDates.DatesPeriod, - scope = NumpyDates.DatesPeriod, - ), - ) - push!(specs, (func = pyconvert_rule_timedelta64, tname = "numpy:timedelta64", type = Missing, scope = Missing)) - push!(specs, (func = pyconvert_rule_timedelta64, tname = "numpy:timedelta64", type = Nothing, scope = Nothing)) - - return specs -end - diff --git a/src/Convert/pandas.jl b/src/Convert/pandas.jl index 356949bc..bf670cc4 100644 --- a/src/Convert/pandas.jl +++ b/src/Convert/pandas.jl @@ -1,15 +1,3 @@ pyconvert_rule_pandas_na(::Type{Nothing}, x::Py) = pyconvert_return(nothing) pyconvert_rule_pandas_na(::Type{Missing}, x::Py) = pyconvert_return(missing) -function pandas_rule_specs() - return PyConvertRuleSpec[ - (func = pyconvert_rule_pandas_na, tname = "pandas._libs.missing:NAType", type = Missing, scope = Any), - ( - func = pyconvert_rule_pandas_na, - tname = "pandas._libs.missing:NAType", - type = Nothing, - scope = Nothing, - ), - ] -end - diff --git a/src/Convert/pyconvert.jl b/src/Convert/pyconvert.jl index bedcaf4c..4853e61c 100644 --- a/src/Convert/pyconvert.jl +++ b/src/Convert/pyconvert.jl @@ -9,7 +9,6 @@ end const PYCONVERT_RULES = Dict{String,Vector{PyConvertRule}}() const PYCONVERT_RULE_ORDER = Ref{Int}(0) const PYCONVERT_EXTRATYPES = Py[] -const PyConvertRuleSpec = NamedTuple{(:func, :tname, :type, :scope),Tuple{Function,String,Type,Type}} """ pyconvert_add_rule(func::Function, tname::String, ::Type{T}, ::Type{S}=T) where {T,S} diff --git a/src/PythonCall.jl b/src/PythonCall.jl index 770f7dcd..7e0e0ca1 100644 --- a/src/PythonCall.jl +++ b/src/PythonCall.jl @@ -86,206 +86,59 @@ function __init__() pyconvert_add_rule(Convert.pyconvert_rule_timedelta, "datetime:timedelta", Nanosecond, Nanosecond) # ctypes rules - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,true}(), "ctypes:c_char", Cchar, Cchar) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,true}(), "ctypes:c_char", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,false}(), "ctypes:c_char", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,false}(), "ctypes:c_char", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,false}(), "ctypes:c_char", Number, Number) - pyconvert_add_rule( - Convert.pyconvert_rule_ctypessimplevalue{Cwchar_t,true}(), - "ctypes:c_wchar", - Cwchar_t, - Cwchar_t, - ) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cwchar_t,true}(), "ctypes:c_wchar", Int, Int) - pyconvert_add_rule( - Convert.pyconvert_rule_ctypessimplevalue{Cwchar_t,false}(), - "ctypes:c_wchar", - Integer, - Integer, - ) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cwchar_t,false}(), "ctypes:c_wchar", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cwchar_t,false}(), "ctypes:c_wchar", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,true}(), "ctypes:c_byte", Cchar, Cchar) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,true}(), "ctypes:c_byte", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,false}(), "ctypes:c_byte", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,false}(), "ctypes:c_byte", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cchar,false}(), "ctypes:c_byte", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuchar,true}(), "ctypes:c_ubyte", Cuchar, Cuchar) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuchar,true}(), "ctypes:c_ubyte", UInt, UInt) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuchar,true}(), "ctypes:c_ubyte", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuchar,false}(), "ctypes:c_ubyte", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuchar,false}(), "ctypes:c_ubyte", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuchar,false}(), "ctypes:c_ubyte", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cshort,true}(), "ctypes:c_short", Cshort, Cshort) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cshort,true}(), "ctypes:c_short", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cshort,false}(), "ctypes:c_short", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cshort,false}(), "ctypes:c_short", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cshort,false}(), "ctypes:c_short", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cushort,true}(), "ctypes:c_ushort", Cushort, Cushort) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cushort,true}(), "ctypes:c_ushort", UInt, UInt) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cushort,true}(), "ctypes:c_ushort", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cushort,false}(), "ctypes:c_ushort", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cushort,false}(), "ctypes:c_ushort", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cushort,false}(), "ctypes:c_ushort", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cint,true}(), "ctypes:c_int", Cint, Cint) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cint,true}(), "ctypes:c_int", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cint,false}(), "ctypes:c_int", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cint,false}(), "ctypes:c_int", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cint,false}(), "ctypes:c_int", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuint,true}(), "ctypes:c_uint", Cuint, Cuint) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuint,true}(), "ctypes:c_uint", UInt, UInt) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuint,true}(), "ctypes:c_uint", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuint,false}(), "ctypes:c_uint", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuint,false}(), "ctypes:c_uint", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cuint,false}(), "ctypes:c_uint", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clong,true}(), "ctypes:c_long", Clong, Clong) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clong,true}(), "ctypes:c_long", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clong,false}(), "ctypes:c_long", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clong,false}(), "ctypes:c_long", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clong,false}(), "ctypes:c_long", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culong,true}(), "ctypes:c_ulong", Culong, Culong) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culong,true}(), "ctypes:c_ulong", UInt, UInt) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culong,false}(), "ctypes:c_ulong", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culong,false}(), "ctypes:c_ulong", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culong,false}(), "ctypes:c_ulong", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culong,false}(), "ctypes:c_ulong", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clonglong,true}(), "ctypes:c_longlong", Clonglong, Clonglong) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clonglong,true}(), "ctypes:c_longlong", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clonglong,false}(), "ctypes:c_longlong", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clonglong,false}(), "ctypes:c_longlong", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Clonglong,false}(), "ctypes:c_longlong", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culonglong,true}(), "ctypes:c_ulonglong", Culonglong, Culonglong) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culonglong,true}(), "ctypes:c_ulonglong", UInt, UInt) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culonglong,false}(), "ctypes:c_ulonglong", Int, Int) - pyconvert_add_rule( - Convert.pyconvert_rule_ctypessimplevalue{Culonglong,false}(), - "ctypes:c_ulonglong", - Integer, - Integer, - ) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culonglong,false}(), "ctypes:c_ulonglong", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Culonglong,false}(), "ctypes:c_ulonglong", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Csize_t,true}(), "ctypes:c_size_t", Csize_t, Csize_t) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Csize_t,true}(), "ctypes:c_size_t", UInt, UInt) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Csize_t,false}(), "ctypes:c_size_t", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Csize_t,false}(), "ctypes:c_size_t", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Csize_t,false}(), "ctypes:c_size_t", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Csize_t,false}(), "ctypes:c_size_t", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cssize_t,true}(), "ctypes:c_ssize_t", Cssize_t, Cssize_t) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cssize_t,true}(), "ctypes:c_ssize_t", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cssize_t,false}(), "ctypes:c_ssize_t", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cssize_t,false}(), "ctypes:c_ssize_t", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cssize_t,false}(), "ctypes:c_ssize_t", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cfloat,true}(), "ctypes:c_float", Cfloat, Cfloat) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cfloat,true}(), "ctypes:c_float", Float64, Float64) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cfloat,false}(), "ctypes:c_float", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cfloat,false}(), "ctypes:c_float", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cdouble,true}(), "ctypes:c_double", Cdouble, Cdouble) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cdouble,true}(), "ctypes:c_double", Float64, Float64) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cdouble,false}(), "ctypes:c_double", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Cdouble,false}(), "ctypes:c_double", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cchar},true}(), "ctypes:c_char_p", Cstring, Cstring) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cchar},true}(), "ctypes:c_char_p", Ptr{Cchar}, Ptr{Cchar}) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cchar},true}(), "ctypes:c_char_p", Ptr, Ptr) - pyconvert_add_rule( - Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cwchar_t},true}(), - "ctypes:c_wchar_p", - Cwstring, - Cwstring, - ) - pyconvert_add_rule( - Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cwchar_t},true}(), - "ctypes:c_wchar_p", - Ptr{Cwchar_t}, - Ptr{Cwchar_t}, - ) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cwchar_t},true}(), "ctypes:c_wchar_p", Ptr, Ptr) - pyconvert_add_rule( - Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cvoid},true}(), - "ctypes:c_void_p", - Ptr{Cvoid}, - Ptr{Cvoid}, - ) - pyconvert_add_rule(Convert.pyconvert_rule_ctypessimplevalue{Ptr{Cvoid},true}(), "ctypes:c_void_p", Ptr, Ptr) + for (t, T) in Convert.CTYPES_SIMPLE_TYPES + name = "ctypes:c_$t" + rule = Convert.pyconvert_rule_ctypessimplevalue{T,false}() + saferule = Convert.pyconvert_rule_ctypessimplevalue{T,true}() + isptr = endswith(t, "_p") + isreal = !isptr + isfloat = t in ("float", "double") + isint = isreal && !isfloat + isuint = isint && (startswith(t, "u") || t == "size_t") + t == "char_p" && pyconvert_add_rule(saferule, name, Cstring, Cstring) + t == "wchar_p" && pyconvert_add_rule(saferule, name, Cwstring, Cwstring) + pyconvert_add_rule(saferule, name, T, T) + isuint && pyconvert_add_rule(sizeof(T) ≤ sizeof(UInt) ? saferule : rule, name, UInt, UInt) + isuint && pyconvert_add_rule(sizeof(T) < sizeof(Int) ? saferule : rule, name, Int, Int) + isint && !isuint && pyconvert_add_rule(sizeof(T) ≤ sizeof(Int) ? saferule : rule, name, Int, Int) + isint && pyconvert_add_rule(rule, name, Integer, Integer) + isfloat && pyconvert_add_rule(saferule, name, Float64, Float64) + isreal && pyconvert_add_rule(rule, name, Real, Real) + isreal && pyconvert_add_rule(rule, name, Number, Number) + isptr && pyconvert_add_rule(saferule, name, Ptr, Ptr) + end # numpy rules - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Bool,true}(), "numpy:bool_", Bool, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Bool,true}(), "numpy:bool_", UInt, UInt) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Bool,true}(), "numpy:bool_", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Bool,false}(), "numpy:bool_", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Bool,false}(), "numpy:bool_", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Bool,false}(), "numpy:bool_", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int8,true}(), "numpy:int8", Int8, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int8,true}(), "numpy:int8", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int8,false}(), "numpy:int8", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int8,false}(), "numpy:int8", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int8,false}(), "numpy:int8", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int16,true}(), "numpy:int16", Int16, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int16,true}(), "numpy:int16", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int16,false}(), "numpy:int16", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int16,false}(), "numpy:int16", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int16,false}(), "numpy:int16", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int32,true}(), "numpy:int32", Int32, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int32,true}(), "numpy:int32", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int32,false}(), "numpy:int32", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int32,false}(), "numpy:int32", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int32,false}(), "numpy:int32", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int64,true}(), "numpy:int64", Int64, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int64,true}(), "numpy:int64", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int64,false}(), "numpy:int64", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int64,false}(), "numpy:int64", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Int64,false}(), "numpy:int64", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt8,true}(), "numpy:uint8", UInt8, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt8,true}(), "numpy:uint8", UInt, UInt) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt8,true}(), "numpy:uint8", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt8,false}(), "numpy:uint8", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt8,false}(), "numpy:uint8", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt8,false}(), "numpy:uint8", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt16,true}(), "numpy:uint16", UInt16, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt16,true}(), "numpy:uint16", UInt, UInt) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt16,true}(), "numpy:uint16", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt16,false}(), "numpy:uint16", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt16,false}(), "numpy:uint16", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt16,false}(), "numpy:uint16", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt32,true}(), "numpy:uint32", UInt32, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt32,true}(), "numpy:uint32", UInt, UInt) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt32,true}(), "numpy:uint32", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt32,false}(), "numpy:uint32", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt32,false}(), "numpy:uint32", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt32,false}(), "numpy:uint32", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt64,true}(), "numpy:uint64", UInt64, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt64,true}(), "numpy:uint64", UInt, UInt) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt64,false}(), "numpy:uint64", Int, Int) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt64,false}(), "numpy:uint64", Integer, Integer) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt64,false}(), "numpy:uint64", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{UInt64,false}(), "numpy:uint64", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float16,true}(), "numpy:float16", Float16, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float16,true}(), "numpy:float16", Float64, Float64) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float16,false}(), "numpy:float16", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float16,false}(), "numpy:float16", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float32,true}(), "numpy:float32", Float32, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float32,true}(), "numpy:float32", Float64, Float64) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float32,false}(), "numpy:float32", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float32,false}(), "numpy:float32", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float64,true}(), "numpy:float64", Float64, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float64,true}(), "numpy:float64", Float64, Float64) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float64,false}(), "numpy:float64", Real, Real) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{Float64,false}(), "numpy:float64", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF16,true}(), "numpy:complex32", ComplexF16, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF16,true}(), "numpy:complex32", ComplexF64, ComplexF64) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF16,false}(), "numpy:complex32", Complex, Complex) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF16,false}(), "numpy:complex32", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF32,true}(), "numpy:complex64", ComplexF32, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF32,true}(), "numpy:complex64", ComplexF64, ComplexF64) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF32,false}(), "numpy:complex64", Complex, Complex) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF32,false}(), "numpy:complex64", Number, Number) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF64,true}(), "numpy:complex128", ComplexF64, Any) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF64,true}(), "numpy:complex128", ComplexF64, ComplexF64) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF64,false}(), "numpy:complex128", Complex, Complex) - pyconvert_add_rule(Convert.pyconvert_rule_numpysimplevalue{ComplexF64,false}(), "numpy:complex128", Number, Number) + for (t, T) in Convert.NUMPY_SIMPLE_TYPES + name = "numpy:$t" + rule = Convert.pyconvert_rule_numpysimplevalue{T,false}() + saferule = Convert.pyconvert_rule_numpysimplevalue{T,true}() + isbool = occursin("bool", t) + isint = occursin("int", t) || isbool + isuint = occursin("uint", t) || isbool + isfloat = occursin("float", t) + iscomplex = occursin("complex", t) + isreal = isint || isfloat + isnumber = isreal || iscomplex + + pyconvert_add_rule(saferule, name, T, Any) + isuint && pyconvert_add_rule(sizeof(T) ≤ sizeof(UInt) ? saferule : rule, name, UInt, UInt) + isuint && pyconvert_add_rule(sizeof(T) < sizeof(Int) ? saferule : rule, name, Int, Int) + isint && !isuint && pyconvert_add_rule(sizeof(T) ≤ sizeof(Int) ? saferule : rule, name, Int, Int) + isint && pyconvert_add_rule(rule, name, Integer, Integer) + isfloat && pyconvert_add_rule(saferule, name, Float64, Float64) + isreal && pyconvert_add_rule(rule, name, Real, Real) + iscomplex && pyconvert_add_rule(saferule, name, ComplexF64, ComplexF64) + iscomplex && pyconvert_add_rule(rule, name, Complex, Complex) + isnumber && pyconvert_add_rule(rule, name, Number, Number) + end pyconvert_add_rule(Convert.pyconvert_rule_datetime64, "numpy:datetime64", NumpyDates.DateTime64, Any) - pyconvert_add_rule(Convert.pyconvert_rule_datetime64, "numpy:datetime64", NumpyDates.InlineDateTime64, NumpyDates.InlineDateTime64) + pyconvert_add_rule( + Convert.pyconvert_rule_datetime64, + "numpy:datetime64", + NumpyDates.InlineDateTime64, + NumpyDates.InlineDateTime64, + ) pyconvert_add_rule( Convert.pyconvert_rule_datetime64, "numpy:datetime64", From b017e45d9367f388fb83e74df66e64c7b677ac03 Mon Sep 17 00:00:00 2001 From: Christopher Rowley Date: Tue, 18 Nov 2025 19:30:06 +0000 Subject: [PATCH 11/11] Clarify conversion changelog entry --- CHANGELOG.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 11a5591e..bb159d80 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,9 +4,9 @@ * The vast majority of these changes are breaking, see the [v1 Migration Guide](@ref) for how to upgrade. * Changes to core functionality: * Comparisons like `==(::Py, ::Py)`, `<(::Py, ::Number)`, `isless(::Number, ::Py)` now return `Bool` instead of `Py`. +* Changes to conversion: * `pyconvert` rules are now scoped by target type instead of prioritized; rules are ordered by Python type specificity and creation order. -### Conversion -* `pyconvert_add_rule` now has the signature `pyconvert_add_rule(func::Function, t::String, ::Type{T}, ::Type{S}=T)`; the optional scope `S` controls when a rule is considered during conversion. + * `pyconvert_add_rule` now has the signature `pyconvert_add_rule(func::Function, t::String, ::Type{T}, ::Type{S}=T)`; the optional scope `S` controls when a rule is considered during conversion. * Changes to `PythonCall.GC` (now more like `Base.GC`): * `enable(true)` replaces `enable()`. * `enable(false)` replaces `disable()`.