Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge from dotnet/runtime #86

Merged
merged 73 commits into from
Sep 8, 2020
Merged

Merge from dotnet/runtime #86

merged 73 commits into from
Sep 8, 2020

Conversation

jkotas
Copy link
Member

@jkotas jkotas commented Sep 8, 2020

No description provided.

gfoidl and others added 30 commits September 2, 2020 19:55
* char.IsAscii xml doc comments

* Update src/libraries/System.Private.CoreLib/src/System/Char.cs

Co-authored-by: Dan Moseley <[email protected]>
…ntime-assets dotnet/xharness dotnet/icu (#41591)

[master] Update dependencies from mono/linker dotnet/arcade dotnet/runtime-assets dotnet/xharness dotnet/icu
- Updates:
  - Microsoft.NET.ILLink.Tasks: from 6.0.0-alpha.1.20424.3 to 6.0.0-alpha.1.20451.1

- Updates:
  - Microsoft.DotNet.GenFacades: from 5.0.0-beta.20427.5 to 5.0.0-beta.20431.4
  - Microsoft.DotNet.Build.Tasks.Feed: from 5.0.0-beta.20427.5 to 5.0.0-beta.20431.4
  - Microsoft.DotNet.Build.Tasks.Packaging: from 5.0.0-beta.20427.5 to 5.0.0-beta.20431.4
  - Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk: from 5.0.0-beta.20427.5 to 5.0.0-beta.20431.4
  - Microsoft.DotNet.Build.Tasks.TargetFramework.Sdk: from 5.0.0-beta.20427.5 to 5.0.0-beta.20431.4
  - Microsoft.DotNet.CodeAnalysis: from 5.0.0-beta.20427.5 to 5.0.0-beta.20431.4
  - Microsoft.DotNet.GenAPI: from 5.0.0-beta.20427.5 to 5.0.0-beta.20431.4
  - Microsoft.DotNet.XUnitExtensions: from 5.0.0-beta.20427.5 to 5.0.0-beta.20431.4
  - Microsoft.DotNet.Helix.Sdk: from 5.0.0-beta.20427.5 to 5.0.0-beta.20431.4
  - Microsoft.DotNet.RemoteExecutor: from 5.0.0-beta.20427.5 to 5.0.0-beta.20431.4
  - Microsoft.DotNet.VersionTools.Tasks: from 5.0.0-beta.20427.5 to 5.0.0-beta.20431.4
  - Microsoft.DotNet.XUnitConsoleRunner: from 2.5.1-beta.20427.5 to 2.5.1-beta.20431.4
  - Microsoft.DotNet.Arcade.Sdk: from 5.0.0-beta.20427.5 to 5.0.0-beta.20431.4
  - Microsoft.DotNet.ApiCompat: from 5.0.0-beta.20427.5 to 5.0.0-beta.20431.4

- Updates:
  - Microsoft.DotNet.XHarness.CLI: from 1.0.0-prerelease.20424.2 to 1.0.0-prerelease.20431.1
  - Microsoft.DotNet.XHarness.TestRunners.Xunit: from 1.0.0-prerelease.20424.2 to 1.0.0-prerelease.20431.1

- Updates:
  - Microsoft.NETCore.Runtime.ICU.Transport: from 6.0.0-alpha.1.20428.1 to 6.0.0-alpha.1.20451.1

- Updates:
  - System.ComponentModel.TypeConverter.TestData: from 5.0.0-beta.20426.1 to 5.0.0-beta.20451.1
  - System.Drawing.Common.TestData: from 5.0.0-beta.20426.1 to 5.0.0-beta.20451.1
  - System.IO.Compression.TestData: from 5.0.0-beta.20426.1 to 5.0.0-beta.20451.1
  - System.IO.Packaging.TestData: from 5.0.0-beta.20426.1 to 5.0.0-beta.20451.1
  - System.Net.TestData: from 5.0.0-beta.20426.1 to 5.0.0-beta.20451.1
  - System.Private.Runtime.UnicodeData: from 5.0.0-beta.20426.1 to 5.0.0-beta.20451.1
  - System.Security.Cryptography.X509Certificates.TestData: from 5.0.0-beta.20426.1 to 5.0.0-beta.20451.1
  - System.Windows.Extensions.TestData: from 5.0.0-beta.20426.1 to 5.0.0-beta.20451.1

 - fix Browser-related tests
* Add ios support to the cmake build system.

* Only link against darwin libs on darwin.

* Add x86_64-apple-darwin10 to offset-tool abis.

* Add aot cross compiler support to the cmake build.

* Disable verifier on the cmake build.

* Add LLVM support.

* Use OS_LIBS in more place.

* Fix the build.
The address of the ind can't be null because it is a prejited location.

However, the value of the indirection could be different, so should not mark as invariant.
In composite build mode, we copy the input assemblies to output,
injecting the component R2R headers into them. This process was
missing the logic to set up the READYTORUN_FLAG_PLATFORM_NEUTRAL_SOURCE
flag, causing subsequent AssemblyName mismatches. At this point
I don't plan to back-propagate the change to .NET 5 as Crossgen2
is not yet generally available and this fixes just a corner case in it.

Thanks

Tomas
* Enable closure validation

* Ensure dependency graph directory exists

* Netstandard should not be ignored

* Ensure Reports dir exists; VerifyClosure change moved to Arcade

* Upgrade SharedFramework.SDK package to pick up a change for VerifyClosure target
Fix some interactions between tiering and multi-core JIT

Issues:
- When a multi-core JIT profile includes a method for which prejitted code is available, and the method contains a loop, when playing back the profile the multi-core JIT of the method disables call counting for the method, then when the method gets called the prejitted code is used but does not get tiered up later. Also hits assertion failures.
- Tier 1 rejits also record the method in the multi-core JIT profile, another case where the above situation can occur

Fixes:
- Only record a method in the multi-core JIT profile for the default code version and only once for the code version
- When playing back a profile, don't disable call counting for a method immediately. Just record some info that may be used later when the method is actually gets called, if the multi-core jitted code gets used, to disable call counting if necessary.
We lost our docs at some point, but the key detail is that prepending COMPlus_ to the knob name is the environment variable you need to set. Hoisting that fact right to the top of the file so that people who come looking can find it.
… default (#41539)

* Add JsonNumberHandling.AllowReadingFromString as a JsonSerializer web default

* Add test assertion for number handling option in S.N.H.Json

* Test number-as-string behavior in System.Net.Http.Json
…ng code randomly crashes without it. (#41798)

Co-authored-by: vargaz <[email protected]>
…40871)

* Add regression test for dotnet/runtime#40607

* Add Runtime_40607.tt

* Add more extensive tests for loads in Runtime_40607.tt Runtime_40607.il

* Enable back failing test in System.Private.Xml.dll

* Fold *(typ*)&lclVar tree when:

1) it is *definitely load* and types of both indirection and local variable have the same signedness (e.g. bool and byte)
2) otherwise, fold the tree and mark the local node with GTF_VAR_FOLDED_IND
   and call fgDoNormalizeOnStore() on such nodes' parents in post-order morph.
…on to keep its definition in cor.h (#41797)

Fixes undefined references to g_tkCorEncodeToken when using CorSigCompressToken()
in user projects with shared Profiling API headers like cor.h / corprof.h

See dotnet/runtime#40254 (comment).

Co-authored-by: Jan Kotas <[email protected]>
String resouces packaged as PE files only exist on Windows.
Reverts "Ensure FileStatus and FileSystemEntry Hidden and ReadOnly attributes are retrieved the same way"
(commit 6ed1e41) which introduced a perf regression.

Co-authored-by: carlossanlop <[email protected]>
* Properly handle byrefs in tailcall helper stubs

Switch to using ByReference instead of using stind.i/ldind.i and relying
on the JIT to report the locations being moved between.

Fixes #41555

* Move NextCallReturnAddress call back
* Clean up DataContractSerialization

- Remove DiagnosticUtility.IsFatal, which always returns false.
- Remove dead CreateDelegate method
- Fix a TODO-NULLABLE where MemberInfo will never be null

* Clean up remaining #if USE_REFEMIT and other unused defines in DataContractSerialization
* [wasm][debugger][tests] Update to use `TDateTime`

- this ensures that we check the datetime, and some property getters on
it, wherever we have a datetime.

* [wasm][debugger][tests] Add labels to more checks

* [wasm][debugger] Add support for surfacing inherited members

- surface inherited fields, and properties
- we try to support `Runtime.getProperties`'s two arguments:
    - `ownProperties`, and `accessorsOnly`

    - `ownProperties`: for JS, this means return only the object's own
    members (not inherited ones)
    - `accessorsOnly`: for JS, this means return all the getters

Actual implementation:

- In practice, VSCode, and Chrome debugger seem to only send
`{ ownProperties: true, accessorsOnly: false }`,
and `{ ownProperties: false, accessorsOnly: true }`. The combination of
which means - that we don't return any inherited fields!

- But we want to show inherited fields too, so to get that behavior we
essentially *ignore* `ownProperties`. IOW,

    - `ownProperties`: we return all fields, and properties
    - `accessorsOnly`: we return only the getters, including the
    inherited ones

- Another thing to note is the case for auto-properties
    - these have a backing field
    - and we usually return the backing field's value, instead of
    returning a getter
    - To continue with that, auto-properties are *not* returned for
    `accessorsOnly`

- The code in `mini-wasm-debugger.c` does handle these two arguments,
but that is currently disabled by not passing the args to debugger.c at
all
    - Instead, we get the *full* list of members, and try to filter it
    in `library_mono.js`
    - which includes handling property overrides, or shadowing by new
    properties/fields in derived classes

* [wasm][debugger][tests] Fix simple warnings

* [wasm][debugger][tests] Fix warnings introduced in this PR

* [wasm][debugger][tests] Fix indentation

* [wasm][debugger] Correctly handle local structs in async methods

- When we have a struct local in an async instance method, it doesn't
get expanded, since we have a containerId (the async object), and we can
expand/access it later.

- When the IDE asks us to expand it with `{accessorPropertiesOnly: true}`:
    - we get the expanded json, but `_filter_automatic_properties` tries
    to return just the accessors, but that doesn't handle the expanded
    members of nested structs!
    - That is done in `extract_and_cache_value_types`, which is run *after*
    `_filter_automatic_properties`, but by that time we have already
    lost the expanded members!

    - So, `_get_vt_properties` fails with `Unknown valuetype id`,
    because it doesn't have anything to return at that point.

- This is being solved by ignoring the getProperties args in case of
expanding valuetypes.
    - that means that we can correctly extract, and cache the whole
    object.
    - And after that, we can return accessors/others, based on the args.

* [wasm][debugger] Fix warnings in debugger-test-app, and turn on warnAsError

* For some cases, debugger seems to give the actual method name instead of MoveNext for async methods
janvorli and others added 26 commits September 5, 2020 01:23
* Fix covariant returns validation

Recent fix to a problem with covariant returns validation has introduced
another problem related to code versioning. The validation method is now
called too late in the type load process which causes problem in
crossgening or in case code versioning is disabled.

This change fixes it by splitting the return type verification and
mutating changes like vtable slot updates into two separate phases.

* Fix missing precondition for calling SetRequiresCovariantReturnTypeChecking
* Add GetBytes, TryWriteBytes, ToHalf overloads for Half.

* Add ref source in System.Runtime.

* Add basic tests.

* Add array and span tests in BitConverter.

* Add xmldoc for BitConverter.GetBytes and TryWriteBytes.

* Add xmldoc for BitConverter.ToXXX.

* Add xmldoc for BitConverter.BitsTo.
* Bundle assemblies at 4K for linux arm64

* Add arch to bundler trace output

* PR feedback

* Remove extra overload

* Update comment

* Fix RID computation
* [wasm][debugger] Add support for Nullable<T>

Return the value, or null.

Fixes mono/mono#20310

* Address review feedback - merge functions

* [wasm][debugger] run dotnet-format on the debugger test app

* [wasm][debugger] simplify function sig, based on usage

- addresses review feedback from @lewing

* [wasm][debugger] Simplify the function further, based on @lewing's

.. excellent suggestion!
* [wasm][debugger] Add support for Nullable<T>

Return the value, or null.

Fixes mono/mono#20310

* Address review feedback - merge functions

* [wasm][debugger] run dotnet-format on the debugger test app

* [wasm][debugger] simplify function sig, based on usage

- addresses review feedback from @lewing

* [wasm][debugger] Simplify the function further, based on @lewing's

.. excellent suggestion!

* [wasm][debugger] Show actual data for boxed values

Eg. `object o = "foobar"`

This will show the string `"foobar"`, instead of an object, in the
debugger.
* [wasm][debugger] Instead of failing completely, skip the problematic

.. property. Some times we might not get a `value`, or `name`, or it
might instead have a `get`. Handle those cases correctly when combining
the name/value/get objects.

This showed up in case of a `MulticastDelegate`, where we didn't have a
`value`, and ended up incorrectly combining the name/value objects, thus
returning incorrect locals.

* [wasm][debugger] Handle MulticastDelegate, and events

- Essentially we just surface these as a symbol showing the type name

* [wasm][debugger] Fail gracefully when vt could not be expanded

* [wasm][debugger] Handle invalid scope ids

scope<0, or scope too high

- This does get filtered at the proxy level, but app side should be able
to handle it too

* [wasm][debugger] Handle invalid/missing/failed value descriptions

- Eg. missing because of invalid param/local id, or value description
failed because of some as yet unsupported type

* [wasm][debugger] Fix frame indexing

- `collect_frames`, `list_frames` - both iterate over stack frames. They
skip some frames. This debug proxy assigns a simple index to each of the
received(filtered) frames.

    - so, if we had `[ frame0, (skipped frame1), frame2 ]`, the proxy will
    have `[ frame0(index:0), frame2(index:1) ]`

- `describe_variables_on_frame` also iterates over stack frames, and
tries to find a given frame by an index. And this index is what the
proxy had assigned.
    - because some frames were skipped, this function also needs to skip
    the *same* ones, so that it can find the intended frame.

    - Instead of trying to keep these in sync, here the indexing is
    changed to be the real index found as we iterate over the frames.
    - And the proxy just uses that.
    - So, we will have `[ frame0(index:0), frame2(index:2) ]`

This showed up in a trace in aspnetcore, which was called via
reflection. And that frame didn't get added to the list because it was
not `MONO_WRAPPER_NONE`, which caused the indexing to get out of sync.

Fixes: dotnet/runtime#41818

* fix warning: remove unused var

* rebase on master, fix errors

* Make frame indices returned from debugger.c, 0-based

- Earlier this 1-based, and it was being adjusted in `MonoProxy`.
- Based on @lewing's suggestion, changing this to be 0-based in
debugger.c, itself, thus removing the need to "fixup" in `MonoProxy`.

* dotnet-format fixes
This PR makes basic blocks a fundamental part of the code generation machinery. Before this change, there wasn't really any notion of basic blocks, there was a single list of instructions. With this change, we emit instructions inside basic blocks which are linked together. These basic blocks are iterated one at a time when doing optimization passes or emitting the final code. Currently, the origin of all basic blocks is IL code (they originate from branch targets and exception clauses) and we might never support creation of additional bblocks (since we could just create more complex instructions, which would also result in faster code). The bblocks will be emitted in the final code in the same order as they appear in IL code. This order is maintained through `next_bb` link.

Since the list of bblocks was completely changed, some fixes were required for debugging support. Some tests seemed to pass by chance, since we didn't emit MINT_SDB_INTR_LOC in the same locations as jit. There are probably still some discrepancies there.

This PR enables inlining of methods with multiple basic blocks. Currently, this might regress performance because we don't merge some basic blocks that we could (for method start), reducing benefits from cprop passes operating on single basic blocks. We will soon add a pass that traverses the list of basic blocks which eliminates dead basic blocks, merging any adjacent ones.

Co-authored-by: BrzVlad <[email protected]>
* add template

* edit servicing doc

* Update PULL_REQUEST_TEMPLATE/servicing_pull_request_template.md

Co-authored-by: Santiago Fernandez Madero <[email protected]>

* Apply suggestions from code review

Co-authored-by: Dan Moseley <[email protected]>
Co-authored-by: Santiago Fernandez Madero <[email protected]>

* add exclude paths

Co-authored-by: Santiago Fernandez Madero <[email protected]>
Co-authored-by: Dan Moseley <[email protected]>
…fset (#40715)

* Implement unsigned reference math in IL.

* Update reference source for Unsafe.

* Copy docs to UIntPtr overloads.

* Add tests for UIntPtr.

* Use nuint instead of UIntPtr.

* Fix reference to CORE_ASSEMBLY.
According to the old (pre- #40476) code and the documentation, we should throw ObjectDisposedException in EndAcceptSocket and EndAcceptTcpClient.

Fixes #41585
- Since we are throwing: `net_http_value_must_be_greater_than, 0` it makes more sense to use the same value in the condition.
- The corner case behavior of Marshal.Realloc method is non-standard and differs between the two realloc methods. Capture the current behavior in tests.
- Add temporary workarounds for Windows vs. Unix differences
- Disable the tests on Mono for now
* Remove multi-targeting from runtime.depproj

and convert it to a csproj and use the NoBuild msbuild sdk. Simplify the
project logic in relation to IL vs native code as well.
* Add transitive CompilerServices.Unsafe dependency

for System.Memory and System.Threading.Tasks.Extensions package
references. Adding the dependency as a P2P to prefer the live asset
over the prebuilt.
There were couple of members / functions for stack limits checking that
were not used anymore.
…mal) (#40345)

Similar to the previous occasion, I have identified this set as the minimum meaningful set of scripts to move without causing too much disruption apart from purely mechanical renames. This change moves the developer-facing test-related CoreCLR scripts under the src/tests folder in accordance with the previous move of the tests themselves:

src/coreclr/build-test.cmd to src/tests/build.cmd
src/coreclr/build-test.sh to src/tests/build.sh
src/coreclr/tests/build.proj to src/tests/build.proj
src/coreclr/tests/runtest.cmd to src/tests/run.cmd
src/coreclr/tests/runtest.sh to src/tests/run.sh
src/coreclr/tests/src/runtest.proj to src/tests/run.proj
src/coreclr/tests/issues.targets to src/tests/issues.targets

The rest of the change constitutes mere path adjustments to match the primary rename.

Fixes: dotnet/runtime#38919

Thanks

Tomas
* Ensure an initial non-concurrent call to OpenLDAP

OpenLDAP requires a single call for initialization before any other concurrent call.

This fixes asserts and segfaults we were seeing when calling OpenLDAP concurrently.
…(#41911)

* Move Marshal memory allocation methods into CoreLib shared partition

* Unify behavior across platforms
@jkotas jkotas mentioned this pull request Sep 8, 2020
@jkotas jkotas merged commit 56fb96c into dotnet:NativeAOT Sep 8, 2020
@jkotas jkotas deleted the runtime-merge branch September 8, 2020 17:00
scalablecory pushed a commit that referenced this pull request Sep 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.