[Xharness] Bring xharness changes to d16-8 #9951
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rojects, and mark it as a .NET project.
…sion behavior for .NET projects. .NET projects will include files from the current directory by default, which means that if we clone the project file and write the cloned project file in a different directory, we'll have to add an automatically included files into the cloned file manually.
…erty to [I]ProcessManager.
.NET projects are vastly simplified, which means that the OutputPath can't be determined by reading the project file itself, it has to be calculated by MSBuild. So that's exactly what we do: we run MSBuild on the project file and get it to print the property we're interested in.
The simulators won't write stdout/stderr to a path with a space.
They end up with the same make targets as non-.NET targets, which prints warnings in the terminal when running make. We're not using the makefile targets much anymore, so postpone implementing them for .NET until we need them for some reason.
…ects. (#9032) * [xharness] Add support for generating a tvOS version of .NET iOS projects. And use it to run the tvOS version of introspection for .NET. * [xharness] Change according to reviews.
…9082) This is slightly faster - ~0.95s vs ~1.4s - (probably because reflection tries to load a lot of other referenced assemblies, which may or may not exist, causing exceptions (if they don't exist) or spend time loading them (which Cecil won't)). It also avoids a lot of exception details showing up when tracing xharness execution.
Grouping simulators means we have to wait until we know which simulators are available, because we group by the simulator's UDID. When running in server mode, we don't need to group simulator test runs, because we run the as the user chooses, not in any particular order. This way the test list loads faster at startup, since we don't have to wait until we've loaded the simulators before we can show the list.
…rin/maccore@2267. (#9103) Apparently a null namespace when using Reflection can be an empty namespace in Cecil. Fixes xamarin/maccore#2267.
This avoids: * Getting the resource stream for every template expansion * Reading the resource stream * A lot of async code. Instead just read the resource stream once, into a string, and return that. This speeds up the startup by ~10% (from ~0.96s to ~0.83s on my machine).
…stead of catching and ignoring. (#9118)
…ce. (#9119) When a process times out, we try to print a stack trace for all threads. This involves executing "lldb --source <script>", where the script contains: process attach --pid 1234 thread list thread backtrace all detach quit Basically we attach to the project, and ask lldb to print stack traces. The problem: 16:09:02.9522580 Printing backtrace for pid=25276 16:09:02.9528060 /usr/bin/lldb --source /var/folders/q7/mkzwrzcn7bzf3g2v38f3c1cw0000gn/T/tmp58e75d85.tmp 16:09:04.6127570 (lldb) command source -s 0 '/var/folders/q7/mkzwrzcn7bzf3g2v38f3c1cw0000gn/T/tmp58e75d85.tmp' 16:09:04.6130020 Executing commands in '/var/folders/q7/mkzwrzcn7bzf3g2v38f3c1cw0000gn/T/tmp58e75d85.tmp'. 16:09:04.6130200 (lldb) process attach --pid 25276 16:09:05.6458750 error: attach failed: Error 1 16:09:05.7529100 25276 Execution timed out after 1200 seconds and the process was killed. 16:09:05.7588770 Execution timed out after 1200 seconds. If any of those commands fail, the subsequent commands aren't executed. This includes the final "quit" command, which means we end up waiting forever for lldb to do its thing, when lldb doesn't think it needs to do anything at all. The fix: pass two different scripts. It turns out subsequent scripts are executed even if previous scripts fail, so we do the equivalent of: lldb --source <attach script> --source <quit script> And now lldb will quit no matter what the attach script does (it still works even if the attach script succeeds, in which case we'll ask lldb to quit twice).
* Log what we're doing. * Wait a bit after sending SIGABRT, to allow for the OS to create any crash reports.
…providing human-readable logs. (#9123) This makes understanding what's going on much easier.
Reading from a file named as the actual content of the watchOS template doesn't _quite_ work. Instead just use the content directly.
…ern and macOS Full. (#9121) * [xharness] Don't generate into the same directory/files for macOS Modern and macOS Full. This meant that we were overwriting some generated files, which meant that we were executing the Modern set of tests instead of the Full set of tests for at least some configurations. * Add a few ignored tests to the System.Configuration test now that we actually run it.
…NET tests. (#9137) It gets too annoying to keep track of which files are where with the project cloning we're doing, if files are implicitly included.
Split the BCL project generation in two: one part that figures out which projects to generate, and which is done synchronously (because we need it to compute the list of tests), then split out the actual generation and run it on a background thread (since that doesn't have to happen until we want to execute those tests). This speeds up launching xharness in server mode significantly (from ~2s to ~0.2s).
Because it's cumbersome to keep the build working when the same directory has multiple project files.
It's not needed anymore.
…le when adding a property. (#9616) Some .NET test projects (monotouch-test) require this to set properties for some test variations, and .NET projects are quite minimal, making it likely that they won't have a particular property that needs to be set/modified. This makes it possible to execute the various monotouch-test variations for .NET in xharness (some of the variations still fail though, so they're still ignored by default).
…release)' variation by default for tvOS. Fixes #xamarin/maccore@2282. (#9659) This test variation ends up being too big (1.5 GB), so it doesn't install properly and the app crashes at startup. So just skip it. Fixes xamarin/maccore#2282.
… BCL. (#9637) * [src] Build the .NET version of our product assemblies using a .NET 5 BCL. We're not shipping the .NET product assemblies in any stable release, so we can use a preview version of the .NET 5 BCL. Also: * Add all the nuget feeds we need to the top-level NuGet.config, even for .NET 5/6, there shouldn't be any conflicts with stable feeds since we use exact version numbers. * Generate a top-level global5.json which is copied to every directory that needs a .NET 5 global.json (overriding the .NET 3.1 global.json in the root directory). * Use the expected dotnet binary during our local build. * [tests] Fix the bgen tests to use .NET 5. * [xharness] Set the current directory to the project directory when running .NET tests. This way we end up using the dotnet version that's configured in global.json for the tests.
…when parsing xml. (#9703) At the very least this makes stepping through code while debugging much faster, since stepping through whitespace and comments is useless.
…ts. Fixes #9400. (#9704) We weren't probably parsing nested test-suite elements in the xml, so implement parsing of test-suite elements using recursion, which is easier and less error-prone (and turns out to work fine too). Also use TextReader/Writer instead of StreamReader/Writer in a few places. This makes it easier to write tests, since TextReader/Writer can use in-memory streams (no need to mess with temporary files). Fixes #9400.
…d away types around for the static registrar. (#9722) The static registrar may need access to types that have been linked away, so store unmarked types so that the static registrar can access them later. This also makes all the monotouch-test variations green, so enable them all. Fixes this monotouch-test when all optimizations are enabled: MonoTouchFixtures.ObjCRuntime.RegistrarTest [FAIL] TestProtocolRegistration : UIApplicationDelegate/17669 Expected: True But was: False at MonoTouchFixtures.ObjCRuntime.RegistrarTest.TestProtocolRegistration() in xamarin-macios/tests/monotouch-test/ObjCRuntime/RegistrarTest.cs:line 1350
…9729) This fixes a build error with a xammac test variation that is executed even if the macOS build is disabled.
…de 12+. (Re)fixes #xamarin/maccore@2284. (#9732) Fixes xamarin/maccore#2284.
mandel-macaque
added
the
pr-change-not-shipping
spouliot
approved these changes
Oct 22, 2020
tj-devel709
approved these changes
Oct 22, 2020
whitneyschmidt
approved these changes
Oct 22, 2020
dalexsoto
approved these changes
Oct 22, 2020
|
Build failure Test results26 tests failed, 81 tests passed.Failed tests
|
rolfbjarne
approved these changes
Oct 23, 2020
|
Build failure Test results24 tests failed, 990 tests' device not found, 123 tests passed.Failed tests
|
|
sim32 failures are known with the current xcode version. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Bring all the changes in xharness, but those in xharness, so that device tests work.