Updating dev #67
Merged
Updating dev #67
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In a few places across Base and the stdlib, we emit paths that we like people to be able to click on in their terminal and editor. Up to this point, we have relied on auto-filepath detection, but this does not allow for alternative link text, such as contracted paths. Doing so (via OSC 8 terminal links for example) requires filepath URI encoding. This functionality was previously part of a PR modifying stacktrace printing (JuliaLang#51816), but after that became held up for unrelated reasons and another PR appeared that would benefit from this utility (JuliaLang#55335), I've split out this functionality so it can be used before the stacktrace printing PR is resolved.
JuliaLang#55466) Each `Module` defined with `module` automatically gets an `include` function with two methods. Each of those two methods takes a file path as its last argument. Even though the path argument is unconstrained by dispatch, it's documented as constrained with `::AbstractString`: https://docs.julialang.org/en/v1.11-dev/base/base/#include Furthermore, I think that any invocation of `include` with a non-`AbstractString` path will necessarily throw a `MethodError` eventually. Thus this change should be harmless. Adding the type constraint to the path argument is an improvement because any possible exception would be thrown earlier than before. Apart from modules defined with `module`, the same issue is present with the anonymous modules created by `evalfile`, which is also addressed. Sidenote: `evalfile` seems to be completely untested apart from the test added here. Co-authored-by: Florian <[email protected]>
Fixes JuliaLang#54203 Requires JuliaLang#55641 Based on JuliaLang#55641 (comment) cc. @JakeZw @ronisbr --------- Co-authored-by: Jameson Nash <[email protected]>
In order to help avoid memory provenance issues, and better utilize stack space (somewhat), and use FCA less, change the preferred representation of an immutable object to be a pair of `<packed-data,roots>` values. This packing requires some care at the boundaries and if the expected field alignment exceeds that of a pointer. The change is expected to eventually make codegen more flexible at representing unions of values with both bits and pointer regions. Eventually we can also have someone improve the late-gc-lowering pass to take advantage of this increased information accuracy, but currently it will not be any better than before at laying out the frame.
…art of the refactoring
…#55863) Since JuliaLang#49110, which is included in 1.10 and 1.11, spawning a task no longer advances the parent task's RNG state, so this statement in the docs was incorrect.
This fixes JuliaLang#54422, the code here assumes that top level exprs are always rooted, but I don't see that referenced anywhere else, or guaranteed, so conservatively always root objects that show up in code.
So easy to type jl_datatype_align to get the natural alignment instead
of julia_alignment to get the actual alignment. This should fix the
Revise workload.
Change is visible with
```
julia> code_llvm(Random.XoshiroSimd.forkRand, (Random.TaskLocalRNG, Base.Val{8}))
```
…iaLang#55884) I thought these sort of problems had been addressed by d60f92c, but it seems some were missed. Specifically, `t.a` and `t.b` from `t::Union` could be `TypeVar`, and if they are passed to a subroutine or recursed without being unwrapped or rewrapped, errors like JuliaLang#55882 could occur. This commit resolves the issue by calling `unwraptv` in the `Union` handling within `instanceof_tfunc`. I also found a similar issue inside `nfields_tfunc`, so that has also been fixed, and test cases have been added. While I haven't been able to make up a test case specifically for the fix in `instanceof_tfunc`, I have confirmed that this commit certainly fixes the issue reported in JuliaLang#55882. - fixes JuliaLang#55882
Just like all other libraries, we don't want internal Julia files to mess with system files. Introduced by JuliaLang#55411.
…g#55826) Additional GC observability tool. This will help us to diagnose why some of our servers are triggering so many full GCs in certain circumstances.
Reverts JuliaLang#55754 as it overrode some performance heuristics which appeared to be giving a significant gain/loss in performance: Closes JuliaLang#55893
…56149) Fixes https://github.com/JuliaLang/julia/issues/56134 After this, ```julia julia> using LinearAlgebra julia> A = hermitianpart(rand(4, 4)) 4×4 Hermitian{Float64, Matrix{Float64}}: 0.387617 0.277226 0.67629 0.60678 0.277226 0.894101 0.388416 0.489141 0.67629 0.388416 0.100907 0.619955 0.60678 0.489141 0.619955 0.452605 julia> B = UpperTriangular(A) 4×4 UpperTriangular{Float64, Hermitian{Float64, Matrix{Float64}}}: 0.387617 0.277226 0.67629 0.60678 ⋅ 0.894101 0.388416 0.489141 ⋅ ⋅ 0.100907 0.619955 ⋅ ⋅ ⋅ 0.452605 julia> B - B' 4×4 Matrix{Float64}: 0.0 0.277226 0.67629 0.60678 -0.277226 0.0 0.388416 0.489141 -0.67629 -0.388416 0.0 0.619955 -0.60678 -0.489141 -0.619955 0.0 ``` This preserves the band structure of the parent, if any: ```julia julia> U = UpperTriangular(Diagonal(ones(4))) 4×4 UpperTriangular{Float64, Diagonal{Float64, Vector{Float64}}}: 1.0 0.0 0.0 0.0 ⋅ 1.0 0.0 0.0 ⋅ ⋅ 1.0 0.0 ⋅ ⋅ ⋅ 1.0 julia> U - U' 4×4 Diagonal{Float64, Vector{Float64}}: 0.0 ⋅ ⋅ ⋅ ⋅ 0.0 ⋅ ⋅ ⋅ ⋅ 0.0 ⋅ ⋅ ⋅ ⋅ 0.0 ``` This doesn't fully work with partly initialized matrices, and would need JuliaLang#55312 for that. The abstract triangular methods now construct matrices using `similar(parent(U), size(U))` so that the destinations are fully mutable. ```julia julia> @invoke B::LinearAlgebra.AbstractTriangular - B'::LinearAlgebra.AbstractTriangular 4×4 Matrix{Float64}: 0.0 0.277226 0.67629 0.60678 -0.277226 0.0 0.388416 0.489141 -0.67629 -0.388416 0.0 0.619955 -0.60678 -0.489141 -0.619955 0.0 ``` --------- Co-authored-by: Daniel Karrasch <[email protected]>
…th memset since aggregate stores are bad (JuliaLang#55879) This fixes the issues found in slack in the reinterprets of ```julia julia> split128_v2(x::UInt128) = (first(reinterpret(NTuple{2, UInt}, x)), last(reinterpret(NTuple{2, UInt}, x))) split128_v2 (generic function with 1 method) julia> split128(x::UInt128) = reinterpret(NTuple{2, UInt}, x) split128 (generic function with 1 method) @code_native split128(UInt128(5)) push rbp mov rbp, rsp mov rax, rdi mov qword ptr [rdi + 8], rdx mov qword ptr [rdi], rsi pop rbp ret @code_native split128_v2(UInt128(5)) push rbp mov rbp, rsp mov rax, rdi mov qword ptr [rdi], rsi mov qword ptr [rdi + 8], rdx pop rbp ret ``` vs on master where ```julia julia> @code_native split128(UInt128(5)) push rbp mov rbp, rsp mov eax, esi shr eax, 8 mov ecx, esi shr ecx, 16 mov r8, rsi mov r9, rsi vmovd xmm0, esi vpinsrb xmm0, xmm0, eax, 1 mov rax, rsi vpinsrb xmm0, xmm0, ecx, 2 mov rcx, rsi shr esi, 24 vpinsrb xmm0, xmm0, esi, 3 shr r8, 32 vpinsrb xmm0, xmm0, r8d, 4 shr r9, 40 vpinsrb xmm0, xmm0, r9d, 5 shr rax, 48 vpinsrb xmm0, xmm0, eax, 6 shr rcx, 56 vpinsrb xmm0, xmm0, ecx, 7 vpinsrb xmm0, xmm0, edx, 8 mov eax, edx shr eax, 8 vpinsrb xmm0, xmm0, eax, 9 mov eax, edx shr eax, 16 vpinsrb xmm0, xmm0, eax, 10 mov eax, edx shr eax, 24 vpinsrb xmm0, xmm0, eax, 11 mov rax, rdx shr rax, 32 vpinsrb xmm0, xmm0, eax, 12 mov rax, rdx shr rax, 40 vpinsrb xmm0, xmm0, eax, 13 mov rax, rdx shr rax, 48 vpinsrb xmm0, xmm0, eax, 14 mov rax, rdi shr rdx, 56 vpinsrb xmm0, xmm0, edx, 15 vmovdqu xmmword ptr [rdi], xmm0 pop rbp ret ```
Currently the header doesn't print for `Pkg.test` with coverage on ``` [8dfed614] Test v1.11.0 1077.2 ms ✓ RequiredInterfaces 1 dependency successfully precompiled in 1 seconds. 8 already precompiled. ```
Adds a new internal function `_take!(dst::Array{T,N}, src::Array{T,N})` for
doing an efficient `copyto!` equivalent. Previously it was assumed that
`compact` did this automatically, which wasn't a great assumption.
Fixes JuliaLang#56078
CacheFlags could get set, but were never propagated to the target process, so the result would be unusable. Additionally, the debug and optimization levels were not synchronized with the sysimg, causing a regression in pkgimage usability after moving out stdlibs. Fixes JuliaLang#56207 Fixes JuliaLang#56054 Fixes JuliaLang#56206
This change by itself doesn't do anything significant on `master`, but when backported to the v1.11 branch it'll address JuliaLang#56177. However it'd be great if someone could tell _why_ this fixes that issue, because it looks very unrelated. --------- Co-authored-by: Ian Butterworth <[email protected]>
…ncy edge tracking (JuliaLang#56179) Disjoint content can be LLVM optimized in parallel now, since codegen no longer has any ability to handle recursion, and compilation should even be able to run in parallel with the GC also. Removes any remaining global state, since that is unsafe. Adds a C++ shim for concurrent gc support in conjunction with using a `std::unique_lock` to DRY code. Fix RuntimeDyld implementation: Since we use the ForwardingMemoryManger instead of making a new RTDyldMemoryManager object every time, we need to reference count the finalizeMemory calls so that we only call that at the end of relocating everything when everything is ready. We already happen to conveniently have a shared_ptr here, so just use that instead of inventing a duplicate counter. Fixes many OC bugs, including mostly fixing JuliaLang#55035, since this bug is just that much harder to express in the more constrained API.
Falling back to the older serial precompilation process is basically a bug (except for if a manifest hasn't been resolved) so JuliaLang#52619 added more info on why it's been hit so we have a chance of fixing issues that are otherwise very difficult to recreate. However "invalid header" which usually just means it was made by a different julia version appears to sound too alarming to users. https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445 So soften it there and in error messages, given it seems a better description. Suggested by @giordano in https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/4?u=ianshmean
…st_throws (JuliaLang#56231) Fix JuliaLang#54082 Arguably this was a breaking change (as a consequence of JuliaLang#51979). But regardless, it seems like useful functionality to have a public API for testing that an `UndefVarError` was thrown for the expected variable name (regardless of scope). This is particularly useful if you don't know what the scope is (for example, in my use-case i want to test that a specific `UndefVarError` is thrown from a module with a `gensym`'d name). Pre-v1.11 the syntax for this was ```julia @test_throws UndefVarError(:x) foo() ``` but that stopped working in v1.11 when `UndefVarError` got a second field (in fact in v1.11.1 this is an error when before it would pass) This PR restores that functionality. We might want to backport it to v1.11.x so that v1.11 isn't the only version that doesn't support this.
This PR contains some refactoring of common functions that were moved to `gc-common.c` and should be shared between MMTk and Julia's stock GC.
qinsoon
reviewed
Oct 20, 2024
…#56257) Stdlib: Pkg URL: https://github.com/JuliaLang/Pkg.jl.git Stdlib branch: master Julia branch: master Old commit: 27c1b1ee5 New commit: 799dc2d54 Julia version: 1.12.0-DEV Pkg version: 1.12.0 Bump invoked by: @IanButterworth Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: JuliaLang/Pkg.jl@27c1b1e...799dc2d ``` $ git log --oneline 27c1b1ee5..799dc2d54 799dc2d54 REPLExt: use Base.isaccessibledir rather than isdir in completions (JuliaLang#4053) 3fde94ee9 REPLExt: run repl hint generation for modeswitch chars when not switching (JuliaLang#4054) ``` Co-authored-by: Dilum Aluthge <[email protected]>
…o! (JuliaLang#56237) This might help llvm understand whats going on. Also enzyme really wants this to look like this to trace through it better. --------- Co-authored-by: Jameson Nash <[email protected]>
This fixes a bug introduced in JuliaLang#55941. We may also take this opportunity to limit the scope of the `@inbounds` annotations, and also use `axes` to compute the bounds instead of hard-coding them. The real "fix" here is on line 767, where `l in 1:nA` should have been `l in 1:mB`. Using `axes` avoids such errors, and makes the operation safer as well.
…iaLang#55984) Currently, `::Diagonal * ::AbstractMatrix` calls the method `LinearAlgebra.__muldiag!` in general that scales the rows, and similarly for the diagonal on the right. The implementation of `__muldiag` was duplicating the logic in `LinearAlgebra.modify!` and the methods for `MulAddMul`. This PR replaces the various branches with calls to `modify!` instead. I've also extracted the multiplication logic into its own function `__muldiag_nonzeroalpha!` so that this may be specialized for matrix types, such as triangular ones. Secondly, `::Diagonal * ::UpperTriangular` (and similarly, other triangular matrices) was specialized to forward the multiplication to the parent of the triangular. For strided matrices, however, it makes more sense to use the structure and scale only the filled half of the matrix. Firstly, this improves performance, and secondly, this avoids errors in case the parent isn't fully initialized corresponding to the structural zero elements. Performance improvement: ```julia julia> D = Diagonal(1:400); julia> U = UpperTriangular(zeros(size(D))); julia> @Btime $D * $U; 314.944 μs (3 allocations: 1.22 MiB) # v"1.12.0-DEV.1288" 195.960 μs (3 allocations: 1.22 MiB) # This PR ``` Fix: ```julia julia> M = Matrix{BigFloat}(undef, 2, 2); julia> M[1,1] = M[2,2] = M[1,2] = 3; julia> U = UpperTriangular(M) 2×2 UpperTriangular{BigFloat, Matrix{BigFloat}}: 3.0 3.0 ⋅ 3.0 julia> D = Diagonal(1:2); julia> U * D # works after this PR 2×2 UpperTriangular{BigFloat, Matrix{BigFloat}}: 3.0 6.0 ⋅ 6.0 ```
qinsoon
approved these changes
Oct 21, 2024
udesou
added a commit
to mmtk/mmtk-julia
that referenced
this pull request
Oct 24, 2024
Backporting #180, merge with mmtk/julia#67.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merging the latest changes from https://github.com/mmtk/julia/tree/upstream-ready/immix (including cleanup/refactoring, and removing the C code from the binding). I've also merged the latest changes from
master.