Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync JsonCodeGen branch with runtime-master #286

Merged
merged 1,666 commits into from
Nov 4, 2020

Conversation

layomia
Copy link

@layomia layomia commented Nov 3, 2020

No description provided.

davidwrighton and others added 30 commits October 14, 2020 17:13
- Fix type layout bugs
  - Sequential or Explicit layout classes without explicit field offsets on arm32 should align their fields based on the start of the field list of the object
  - The field base offset used for R2R calculation on Arm32 should respect the RequiresAlign8 flag
  - Computing true for requiresAlign8 in auto field layout should set the alignment of a class to 8 during auto layout
  - if a class derives from an type which requires 8 byte alignment, set the derived to require higher alignment
- Align the EH info table on 4 byte boundaries
- Set the thumb bit on the arm32 personality routine RVA in XData
- Enable Crossgen2 smoke test for arm
- Adjust architecture specific type layout tests to match CoreCLR behavior
- Fix alignment of Export functions within PE file
…es (#43387)

* [browser][crypto] Remove restraining not supported attribute

- The modules included within the System.Security.Cryptography.Primitives module should still be available for use outside of browser os.

* Address review comments.  Remove `<IncludePlatformAttributes>` attribute as well
* Clean up DependencyContextWriter

- Remove UnifiedJsonWriter
- Remove ArrayBufferWriter and write to the Stream directly

* Clean up DependencyContextJsonReader

- Remove UnifiedJsonReader
- Move any reader logic to extension methods
Currently mono desktop runtime packs don't include `libhostfxr.*` and `libhostpolicy*` libs needed for corehost in self contained mode.

Co-authored-by: Alexander Köplinger <[email protected]>
* Remove some unsafe code from Console

* preserve original byte-by-byte decoding

* remove unsafe declaration where not needed

Co-authored-by: Stephen Toub <[email protected]>
* Bind byte array from base64 config value

* Add test case for failure

* Add test for null case

* Remove unnecessary null check

Co-authored-by: Thomas Levesque <[email protected]>
* [runtime] Add an options API.

Add a general options API to the runtime, based on the flags API in Google V8:

```https://chromium.googlesource.com/v8/v8.git/+/refs/heads/master/src/flags/```

Supported features:
* Definition of runtime options in a declarative way.
* Options are mapped to C globals.
* BOOL/INT/STRING data types.
* Generic option parsing code.
* Generic usage code.
* Read-only flags for build-time optimization.

This is designed to replace the many option parsing functions in
the runtime, MONO_DEBUG, the many mono_set_... functions etc.

* Fix the build.

Co-authored-by: vargaz <[email protected]>
Co-authored-by: Zoltan Varga <[email protected]>
…n Windows (#42427)" (#43442)

This reverts commit 51f6b8b.
After some recent shuffles of tests location, running the coreclr tests
with runincontext option stopped working. This change fixes it by fixing
the script path in the run.py.
* Debugger.IsAttached is now working on wasm. And can be used to detect if debugger is attached.
Fix #42411

* Update src/mono/wasm/runtime/library_mono.js

Co-authored-by: Ryan Lucia <[email protected]>

* Using the infrastructure to not send dynamically loaded assemblies if debugger is not attached.
Changing where to check if the assembly is already added to avoid unnecessary checks.

* Checking the assembly name size.

Co-authored-by: Ryan Lucia <[email protected]>
* Update README table generator.
* Link README to new table location.
* Add 6.0 coreclr runtime links for table generation.
* Add generated table to the dogfooding page.
* Update some stale references to sleet feeds, older versions of the runtime, and deprecated packages.
* Add subset for table generation and reorder table to frontload OS groups.

Co-authored-by: Jan Kotas <[email protected]>
Expressions like `~(BBF_KEEP_BBJ_ALWAYS)` were being evaluated as 32 bit signed
quantities, leading to mask value `00000000_7FFFFFFFF` instead of the desired
`FFFFFFFF_7FFFFFFFF`, causing inadvertent clearing of flags with higher value.

SPMI diffs showed the only flag loss that impacted codegen was `BBF_HAS_CALL`,
which feeds into the CSE heuristics. So no known correctness issue, but it is
certainly possible to also lose `BBF_DOMINATED_BY_EXCEPTIONAL_ENTRY` or
`BBF_HAS_SUPPRESSGC_CALL` and that may be more serious.
* Disable two tests for runincontext testing

These tests are not compatible with running using the runincontext tool.

* Reflect PR feedback - make the test work instead
Because the PKCS#12/PFX import is now done with managed code, the PKCS12 shim was dead code.
1. If we're inheriting a fraction of the profile weight of a profiled block,
mark the inheriting block as profiled. This prevents methods like
`optSetBlockWeights` or `optMarkLoopBlocks` from coming along later and setting
the weights to something else. Since the full inheritance method has similar
logic, make it delegate to the fractional one, with a scale of 100 (no scaling).

2. If we switch from Tier0 to FullOpt, make sure to clear the BBINSTR flag,
else we'll put probes into optimized code.

3. Dump edge weights in the dot graph, if we have them.

4. Only dump the flow graph twice per phase.
* adjust renegotiation tests to match product change

* add assert for validationCount
This baselines the remaining test failures with GH issues to track for further investigation.
* Fix CA1416 warnings in runtime repo
…#43134)

* Add methods to ref file.

* Add empty methods to src files.

* Add the .NET Framework version of these methods for .NET Standard.

* Move OpenExistingResult enum to Common and consume it where needed.

* Remove Unix comment in Windows-only file.

* Document OpenExistingResult enum.

* Make out result parameters nullable.

* Add exception resource string.

* Imlement EventWaitHandleAcl methods.

* Implement MutexAcl methods.

* Implement SemaphoreAcl methods.

* Remove unnecessary check for null or empty name.

* Document the EventWaitHandleAcl methods.

* Document the MutexAcl methods.

* Document the SemaphoreAcl methods.

* Add negative enum check. Fix incorrect cast.

* Add NotNullWhen attribute to TryOpenExisting out parameter. Adjust docs.

* Add EventWaitHandle basic unit tests.

* Add Semaphore basic unit tests

* Add Mutex basic unit tests.

* Add Mutex exception handling unit tests.

* Add EventWaitHandle exception handling unit tests.

* Add Semaphore exception handling unit tests.

* EventWaitHandle and Mutex throw DirectoryNotFoundException when PathNotFound. Adjust documentation.

* Nullability in ref file.

* Do not check for rights out of range value, let Windows handle it. Adjust unit tests accordingly.

* Spacing.

* Remove enum range test. Add PathNotFound tests.

Co-authored-by: Carlos Sanchez Lopez <[email protected]>
We don't want most analyzers running over our test code currently (some rules could be enabled with varying degrees of effort), but we do want the xunit analyzers running, and they haven't been.  Fix that by creating a new ruleset specific to library tests, and switching over to use it when building library test projects.
dotnet-maestro bot and others added 21 commits October 31, 2020 21:38
…otnet/xharness (#44062)

[master] Update dependencies from dotnet/arcade dotnet/llvm-project dotnet/xharness
Also, improve test coverage for FileStream.SetLength
* Support IList<string>

* Revert API change

* Feedback

* Use ReadOnlySpan

* Feedback

* Fix comment

* address feedback

* Add test for List<string> optimization

* Remove StartIndex and Count in core

* Update src/libraries/System.Private.CoreLib/src/System/String.Manipulation.cs

Co-authored-by: Stephen Toub <[email protected]>
- Fix issue where method token referred to derived type but method was actually defined on base type
  - Previously the methods did not carry enough state to determine when to emit the owner type of the method, and which exact type was the owning type. The new logic computes it from the token in the presence of a proper instantiation context, which allows for correct operation.

- Fix issue where constrained dispatch on a method of a valuetype would not put in the correct owner type
  - Issue fixed by determining if the token resolves to a method on the same type as the eventual target method, or if it needs to have a specific owning type specified

In general these issues where both caused by confusion around exactly the correct owning type, and it turned out that computation could not be computed within the signature emitter code, but instead needed to be computed in the JIT at point of use. Fortunately, we had a structure `MethodWithToken` that is used in these (and only these situations). Finally, I also made a pass through the emitter and related logic to remove various band-aids that had built up over the last few years to make all the tests and applications pass. I believe that the new logic should be correct in the general case.

Bonus tweak...  Use parallelism when compiling the framework composite images with crossgen2, and fix bug in composite image generation where mangled symbol names might conflict.

Fixes #43466 and fixes #43467
* Remove unsafe code from String.Join

- Use a span for the separator to share the same code across a char separator and string separator, rather than using pointers.
- Consolidate argument validation for the value/startIndex/count overloads into one shared helper
- Change indexing to use standard 0 to values.Length for loop to eliminate bounds checking on span accesses

* Update src/libraries/System.Private.CoreLib/src/System/String.Manipulation.cs
* Remove size_t from managed Brotli code

* Apply suggestions from code review

Co-authored-by: Stephen Toub <[email protected]>

Co-authored-by: Stephen Toub <[email protected]>
…133)

C#, IL, and asm are all a bit smaller.  We don't need to set the arguments to Empty if they're null, as the called function checks for null.
* Set GC_SUSPEND based on the target os not the host os.

* Add ENABLE_LLVM_RUNTIME cmake option.
…builds. (#41897)

* Move DacTableGen out of the CMake build and into tools-local.

* Attempt: usenmakefiles means ninja.

* Make changes to Windows build to support non-VS generators like ninja.

* Use CMake 3.16 PCHs.

* Remove explicit process count.

* Build and run DacTableGen via the .NET CLI we pull down for the build instead of via CMake.

* Update configurecompiler.cmake to specify language (needed for Ninja).

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Get Ninja build working again.

* Pass configuration to cmake configure step for Ninja build.

* Rename flag

* Fix configure step for cross-arch native build

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Support building via Ninja for coreclr via an MSBuild property

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Implement support for opening the CoreCLR solution in VS via the -vs flag.

* Fix setting generator to Ninja.

* Merge libraries gen-buildsys-win.bat with runtime gen-buildsys.cmd to share windows cmake configure infra.

* Enable building libraries native build with ninja.

* Centralize setting CMAKE_SYSTEM_VERSION for windows builds.

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Update host build to use shared gen-buildsys.cmd

* Fix multi-config generator detection.

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Exit with success when we build successfully.

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Enable and fix up Ninja build for hosts.

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Only link delayimp.lib on Windows

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Remove "overriding /W3 with /W1" warning in Ninja build of hosts by turning on /W3 and disabling any new warnings we hit.

* Reduce command line override warnings by adopting the MSVC_RUNTIME_LIBRARY abstraction feature in CMake 3.14+ (below the minimum requirement on Windows).

* Use the delayed expansion syntax to correctly pass in the config to cmake.

* Move all usages of MSVC runtime library selection to the CMake abstraction to reduce console spew about overridden flags.

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Condition the rest of the compiler options correctly so they don't show up as invalid parameter warnings for assembly builds.

* Correctly build DIALib and DacTableGen as AnyCPU

* Fix WRITE_BARRIER_CHECK define

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Fix passing arch for MSBuild build.

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Configure ASM compiler command line format in configurecompiler.cmake and use CMake's assembly compilation step to build ASM files for ARM/ARM64 instead of manually building them and adding the generated files as additional sources.

* Cleanup after fixing ARM/ARM64 support.

* Rename variable

* Condition PCH's for C++-only as needed to fix using them on non-Windows (where we compile C as C instead of as C++).

* Fix CoreCLR native test build on windows to point to new gen-buildsys location.

* Add missing compilation options in Ninja build that are implicitly defined in the MSBuild build.

* Disable analyzers for DacTableGen. Remove /MAP linker flag (that's for local testing).

* Upgrade to CMake 3.16.4

* Don't warn on unused variables in CMake (these usually vary on platform/config)

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Manually compile ARM and ARM64 asm files when using the VS generator since it doesn't support emitting MARMASM item types in projects (and as a result cannot compile ARM or ARM64 asm without assistance).

* Enable CMake policy in test build to use CMAKE_MSVC_RUNTIME_LIBRARY variable.

* Update initially disabled warnings in host.

* Add -ninja arg to root scripts to enable using Ninja.

* Enable using Ninja in CoreCLR CI.

* Try to fix passing the -ninja arg in yaml.

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Add ninja to the Windows invocation

* Fix 2-phase cross targeting to do one phase at a time and initialize the VC++ environment for the correct build tools each time instead of interleaving them and relying on Visual Studio to handle the differing targets.

* Restore old /Ox optimization setting for Release.

* Fix typos, don't pass exception flags to C files.

* Remove unneeded pragma.

* Fix .pgd file installation.

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Add mention of the `-ninja` flag in the Building CoreCLR documentation.

Signed-off-by: Jeremy Koritzinsky <[email protected]>

* Mention installation locations for Ninja and warn while building if CMake is older than 3.16.0.

* Update docs/workflow/requirements/windows-requirements.md
… pinning (#44118)

* Delete duplicate Buffer.Memmove implementations and avoid unnecessary pinning

We have several versions of Buffer.Memmove, including one implemented around `byte*` and one implemented around `ref byte`.  We can delete the former and just use the latter everywhere.  In doing so, we can also remove a few wrapper helpers, and reduce pinning in places where those wrappers were pinning to get pointers only to eventually end up in the ref-based implementation, anyway.

* Use ThrowHelper in more places
… (#42282)

* Bug fix and unit test

* Address comments

* An elegant fix and unit tests

* Remove old code

* Fix the unit tests

* sq

* sq

* Address comments and add test case

* Skip the unit tests on net framework

* Debug commit for CI

* Try to get more error info

* Run validation as a unit test.

Skip it on non-ICU environments
* Add NativeMainSource alternative to Android Sample

* Move nativeMainSource from parameter to property

Co-authored-by: Mitchell Hwang <[email protected]>
@layomia layomia self-assigned this Nov 3, 2020
@layomia layomia merged commit ed95d1d into dotnet:feature/JsonCodeGen Nov 4, 2020
@layomia layomia deleted the SyncWithMaster branch November 10, 2020 06:29
@joperezr joperezr mentioned this pull request Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment