Merge alpha into beta

.github/CODEOWNERS

@@ -0,0 +1,46 @@
+# This is a comment.
+# Each line is a file pattern followed by one or more owners.
+
+# These owners will be the default owners for everything in
+# the repo. Unless a later match takes precedence,
+# @global-owner1 and @global-owner2 will be requested for
+# review when someone opens a pull request.
+# *       @global-owner1 @global-owner2
+
+# Order is important; the last matching pattern takes the most
+# precedence. When someone opens a pull request that only
+# modifies JS files, only @js-owner and not the global
+# owner(s) will be requested for a review.
+# *.js    @js-owner
+
+# You can also use email addresses if you prefer. They'll be
+# used to look up users just like we do for commit author
+# emails.
+# *.go docs@example.com
+
+# In this example, @doctocat owns any files in the build/logs
+# directory at the root of the repository and any of its
+# subdirectories.
+# /build/logs/ @doctocat
+
+# The `docs/*` pattern will match files like
+# `docs/getting-started.md` but not further nested files like
+# `docs/build-app/troubleshooting.md`.
+# docs/*  docs@example.com
+
+# In this example, @octocat owns any file in an apps directory
+# anywhere in your repository.
+# apps/ @octocat
+
+# In this example, @doctocat owns any file in the `/docs`
+# directory in the root of your repository and any of its
+# subdirectories.
+# /docs/ @doctocat
+
+# Start of NVDA config
+
+# By default auto request review from NV Access developer team.
+* @nvaccess/developers
+
+# For changes to the userGuide auto request review from userDocs team
+/user_docs/en/userGuide.t2t @nvaccess/userDocs

.github/CONTRIBUTING.md

@@ -0,0 +1,4 @@
+The NVDA project is open to contributions, thank you for your interest!
+
+Please first [read our guide to contributing](https://github.com/nvaccess/nvda/wiki/Contributing) on the NVDA
+GitHub Wiki.

.github/ISSUE_TEMPLATE.md

@@ -10,6 +10,8 @@ Please thoroughly read NVDA's wiki article on how to fill in this template, incl
 Issues may be closed if the required information is not present.
 https://github.com/nvaccess/nvda/wiki/Github-issue-template-explanation-and-examples
 Please also note that the NVDA project has a Citizen and Contributor Code of Conduct which can be found at https://github.com/nvaccess/nvda/blob/master/CODE_OF_CONDUCT.md. NV Access expects that all contributors and other community members read and abide by the rules set out in this document while participating or contributing to this project. This includes creating or commenting on issues and pull requests. 
+
+Each of the questions and sections below start with multiple hash symbols (#). Place your answers and information on the blank line below each question.
 -->
 
 ### Steps to reproduce:
@@ -34,6 +36,6 @@ Please also note that the NVDA project has a Citizen and Contributor Code of Con
 
 #### Have you tried any other versions of NVDA? If so, please report their behaviors.
 
-#### If addons are disabled, is your problem still occuring?
+#### If add-ons are disabled, is your problem still occurring?
 
-#### Did you try to run the COM registry fixing tool in NVDA menu / tools?
+#### Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

.github/ISSUE_TEMPLATE/bug_report.md

@@ -9,6 +9,8 @@ Please thoroughly read NVDA's wiki article on how to fill in this template, incl
 Issues may be closed if the required information is not present.
 https://github.com/nvaccess/nvda/wiki/Github-issue-template-explanation-and-examples
 Please also note that the NVDA project has a Citizen and Contributor Code of Conduct which can be found at https://github.com/nvaccess/nvda/blob/master/CODE_OF_CONDUCT.md. NV Access expects that all contributors and other community members read and abide by the rules set out in this document while participating or contributing to this project. This includes creating or commenting on issues and pull requests. 
+
+Each of the questions and sections below start with multiple hash symbols (#). Place your answers and information on the blank line below each question.
 -->
 
 ### Steps to reproduce:
@@ -33,6 +35,6 @@ Please also note that the NVDA project has a Citizen and Contributor Code of Con
 
 #### Have you tried any other versions of NVDA? If so, please report their behaviors.
 
-#### If addons are disabled, is your problem still occuring?
+#### If add-ons are disabled, is your problem still occurring?
 
-#### Did you try to run the COM registry fixing tool in NVDA menu / tools?
+#### Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1 @@
+blank_issues_enabled: false

.github/ISSUE_TEMPLATE/feature_request.md

@@ -9,6 +9,8 @@ Please thoroughly read NVDA's wiki article on how to fill in this template, incl
 Issues may be closed if the required information is not present.
 https://github.com/nvaccess/nvda/wiki/Github-issue-template-explanation-and-examples
 Please also note that the NVDA project has a Citizen and Contributor Code of Conduct which can be found at https://github.com/nvaccess/nvda/blob/master/CODE_OF_CONDUCT.md. NV Access expects that all contributors and other community members read and abide by the rules set out in this document while participating or contributing to this project. This includes creating or commenting on issues and pull requests. 
+
+Each of the questions and sections below start with multiple hash symbols (#). Place your answers and information on the blank line below each question.
 -->
 
 ### Is your feature request related to a problem? Please describe.

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,6 +1,8 @@
 <!-- Please read and fill in the following template, for an explanation of the sections see:
 https://github.com/nvaccess/nvda/wiki/Github-pull-request-template-explanation-and-examples
 Please also note that the NVDA project has a Citizen and Contributor Code of Conduct which can be found at https://github.com/nvaccess/nvda/blob/master/CODE_OF_CONDUCT.md. NV Access expects that all contributors and other community members read and abide by the rules set out in this document while participating or contributing to this project. This includes creating or commenting on issues and pull requests. 
+
+Please initially open PRs as a draft. See https://github.com/nvaccess/nvda/wiki/Contributing
 -->
 
 ### Link to issue number:
@@ -9,11 +11,30 @@ Please also note that the NVDA project has a Citizen and Contributor Code of Con
 
 ### Description of how this pull request fixes the issue:
 
-### Testing performed:
+### Testing strategy:
 
 ### Known issues with pull request:
 
-### Change log entry:
+### Change log entries:
+New features
+Changes
+Bug fixes
+For Developers
+
+### Code Review Checklist:
 
-Section: New features, Changes, Bug fixes
+<!--
+This checklist is a reminder of things commonly forgotten in a new PR.
+Authors, please do a self-review and confirm you have considered the following items.
+Mark items you have considered by checking them.
+You can do this when editing the Pull request description with an x: `[ ]` becomes `[x]`.
+You can also check the checkboxes after the PR is created.
+-->
 
+- [ ] Pull Request description is up to date.
+- [ ] Unit tests.
+- [ ] System (end to end) tests.
+- [ ] Manual tests.
+- [ ] User Documentation.
+- [ ] Change log entry.
+- [ ] Context sensitive help for GUI changes.

.gitignore

@@ -24,20 +24,24 @@ dlldata.c
 *.pdb
 .sconsign.dblite
 user_docs/*/*.html
+user_docs/*/*.css
+user_docs/*/keyCommands.t2t
+user_docs/build.t2tConf
 extras/controllerClient/x86/nvdaController.h
 extras/controllerClient/x64
 extras/controllerClient/arm64
 source/_buildVersion.py
-user_docs/*/keyCommands.t2t
 output
 testOutput
-user_docs/build.t2tConf
 launcher/nvda_logo.wav
 uninstaller/UAC.nsh
 *.pyc
 *.pyo
 *.dmp
 tests/unit/nvda.ini
+tests/system/settingsCache/*
+!tests/system/settingsCache/2020.4/*.txt
 source/locale/*/cldr.dic
 .vscode
 .vs
+.venv

.gitmodules

@@ -15,31 +15,15 @@
 	path = include/sonic
 	url = https://github.com/waywardgeek/sonic.git
 	ignore = untracked
-[submodule "include/pyserial"]
-	path = include/pyserial
-	url = https://github.com/pyserial/pyserial.git
-	ignore = untracked
-[submodule "include/comtypes"]
-	path = include/comtypes
-	url = https://github.com/nvaccess/comtypes-bin
-[submodule "include/scons"]
-	path = include/scons
-	url = https://github.com/SCons/scons
 [submodule "include/ia2"]
 	path = include/ia2
 	url = https://github.com/LinuxA11y/IAccessible2.git
-[submodule "include/wxPython"]
-	path = include/wxPython
-	url = https://github.com/nvaccess/wxPython-bin.git
-[submodule "include/configobj"]
-	path = include/configobj
-	url = https://github.com/DiffSK/configobj.git
-[submodule "include/py2exe"]
-	path = include/py2exe
-	url = https://github.com/nvaccess/py2exe-bin
 [submodule "include/javaAccessBridge32"]
 	path = include/javaAccessBridge32
 	url = https://github.com/nvaccess/javaAccessBridge32-bin.git
+[submodule "include/nvda_dmp"]
+	path = include/nvda_dmp
+	url = https://github.com/codeofdusk/nvda_dmp
 [submodule "include/w3c-aria-practices"]
 	path = include/w3c-aria-practices
 	url = https://github.com/w3c/aria-practices

appveyor.yml

@@ -56,8 +56,6 @@ init:
       Set-AppveyorBuildVariable "versionType" $versionType
      }
 
-clone_depth: 1
-
 install:
  - cd appveyor
  # Decrypt files.
@@ -95,12 +93,20 @@ build_script:
      Set-AppveyorBuildVariable "sconsOutTargets" $sconsOutTargets
      Set-AppveyorBuildVariable "sconsArgs" $sconsArgs
  - 'echo scons args: %sconsArgs%'
- - py scons.py source %sconsArgs%
+ - scons source %sconsArgs%
+ - if ERRORLEVEL 1 exit %ERRORLEVEL%
  # We don't need launcher to run checkPot, so run the checkPot before launcher.
- - py scons.py checkPot %sconsArgs%
+ - ps: |
+     cmd.exe /c "scons checkPot $sconsArgs"
+     if($LastExitCode -ne 0) {
+      $errorCode=$LastExitCode
+      Add-AppveyorMessage "Translation comments missing or unexpectedly included."
+     }
+     if ($errorCode -ne 0) { $host.SetShouldExit($errorCode) }
  # The pot gets built by tests, but we don't actually need it as a build artifact.
  - 'echo scons output targets: %sconsOutTargets%'
- - py scons.py %sconsOutTargets% %sconsArgs%
+ - scons %sconsOutTargets% %sconsArgs%
+ - if ERRORLEVEL 1 exit %ERRORLEVEL%
  # Build symbol store.
  - ps: |
      foreach ($syms in
@@ -115,12 +121,10 @@ build_script:
      }
  - cd symbols
  - 7z a -tzip -r ..\output\symbols.zip *.dl_ *.ex_ *.pd_
+ - if ERRORLEVEL 1 exit %ERRORLEVEL%
  - cd ..
 
 before_test:
- # install required packages
- - py -m pip install --upgrade pip
- - py -m pip install -r tests/system/requirements.txt -r tests/lint/lintInstall/requirements.txt
  - mkdir testOutput
  - mkdir testOutput\unit
  - mkdir testOutput\system
@@ -132,6 +136,7 @@ before_test:
       $nvdaLauncherFile+="_snapshot"
      }
      $nvdaLauncherFile+="_${env:version}.exe"
+     Set-AppveyorBuildVariable "nvdaLauncherFile" $nvdaLauncherFile
      echo NVDALauncherFile: $NVDALauncherFile
      $outputDir=$(resolve-path .\testOutput)
      $installerLogFilePath="$outputDir\nvda_install.log"
@@ -145,6 +150,12 @@ before_test:
       Add-AppveyorMessage "Unable to install NVDA prior to tests."
      }
      Push-AppveyorArtifact $installerLogFilePath
+     $crashDump = "$outputDir\nvda_crash.dmp"
+     if (Test-Path -Path $crashDump){
+      Push-AppveyorArtifact $crashDump -FileName "nvda_install_crash.dmp"
+      Add-AppveyorMessage "Install process crashed"
+      $errorCode=1
+     }
      if($errorCode -ne 0) { $host.SetShouldExit($errorCode) }
 
 test_script:
@@ -153,7 +164,7 @@ test_script:
      $errorCode=0
      $outDir = (Resolve-Path .\testOutput\unit\)
      $unitTestsXml = "$outDir\unitTests.xml"
-     py -m nose -sv --with-xunit --xunit-file="$unitTestsXml" ./tests/unit
+     .\rununittests.bat --with-xunit --xunit-file="$unitTestsXml"
      if($LastExitCode -ne 0) {
       $errorCode=$LastExitCode
       Add-AppveyorMessage "Unit test failure"
@@ -165,18 +176,28 @@ test_script:
 
 # Flake8 Linting
  - ps: |
-    if($env:APPVEYOR_PULL_REQUEST_NUMBER) {
+    if ($env:APPVEYOR_PULL_REQUEST_NUMBER -or $env:APPVEYOR_REPO_BRANCH.StartsWith("try-")) {
       $lintOutput = (Resolve-Path .\testOutput\lint\)
       $lintSource = (Resolve-Path .\tests\lint\)
-      git fetch -q origin $env:APPVEYOR_REPO_BRANCH
-      $prDiff = "$lintOutput\prDiff.patch"
-      git diff -U0 FETCH_HEAD...HEAD > $prDiff
-      $flake8Config = "$lintSource\flake8.ini"
-      $flake8Output = "$lintOutput\PR-Flake8.txt"
-      type "$prDiff" | py -Xutf8 -m flake8 --diff --output-file="$flake8Output" --tee --config="$flake8Config"
+      $flake8Output = "$lintOutput\Flake8.txt"
+      # When Appveyor runs for a pr,
+      # the build is made from a new temporary commit,
+      # resulting from the pr branch being merged into its base branch.
+      # Therefore to create a diff for linting, we must fetch the head of the base branch.
+      # In a PR, APPVEYOR_REPO_BRANCH points to the head of the base branch. 
+      # Additionally, we can not use a clone_depth of 1, but must use an unlimited clone.
+      if($env:APPVEYOR_PULL_REQUEST_NUMBER) {
+        git fetch -q origin $env:APPVEYOR_REPO_BRANCH
+        $msgBaseLabel = "PR"
+      } else {
+        # However in a pushed branch, we must fetch master.
+        git fetch -q origin master:master
+        $msgBaseLabel = "Branch"
+      }
+      .\runlint.bat FETCH_HEAD "$flake8Output" 
       if($LastExitCode -ne 0) {
-       $errorCode=$LastExitCode
-       Add-AppveyorMessage "PR introduces Flake8 errors"
+        $errorCode=$LastExitCode
+        Add-AppveyorMessage "$msgBaseLabel introduces Flake8 errors"
       }
       Push-AppveyorArtifact $flake8Output
       $junitXML = "$lintOutput\PR-Flake8.xml"
@@ -191,19 +212,13 @@ test_script:
  - ps: |
      $testOutput = (Resolve-Path .\testOutput\)
      $systemTestOutput = (Resolve-Path "$testOutput\system")
-     $testSource = ".\tests\system\robot"
-
-     $robotArgs = '--argumentfile', '.\tests\system\robotArgs.robot'
-     # useInstalledNVDA must come after args file to override the whichNVDA value
-     $useInstalledNVDA = '--variable', 'whichNVDA:installed'
-
-     py -m robot $robotArgs $useInstalledNVDA $testSource
-     Compress-Archive -Path "$systemTestOutput\*" -DestinationPath "$testOutput\systemTestResult.zip"
-     Push-AppveyorArtifact "$testOutput\systemTestResult.zip"
+     .\runsystemtests.bat --variable whichNVDA:installed --variable installDir:"${env:nvdaLauncherFile}" --include installer
      if($LastExitCode -ne 0) {
       $errorCode=$LastExitCode
       Add-AppveyorMessage "System test failure"
      }
+     Compress-Archive -Path "$systemTestOutput\*" -DestinationPath "$testOutput\systemTestResult.zip"
+     Push-AppveyorArtifact "$testOutput\systemTestResult.zip"
      $wc = New-Object 'System.Net.WebClient'
      $wc.UploadFile("https://ci.appveyor.com/api/testresults/junit/$($env:APPVEYOR_JOB_ID)", (Resolve-Path "$systemTestOutput\systemTests.xml"))
      if($errorCode -ne 0) { $host.SetShouldExit($errorCode) }
@@ -252,7 +267,13 @@ deploy_script:
        exe=$exe; hash=$hash;
        artifacts=$artifacts
       }
-      ConvertTo-Json -InputObject $data -Compress | ssh nvaccess@exbi.nvaccess.org nvdaAppveyorHook
+      ConvertTo-Json -InputObject $data -Compress | Out-File -FilePath deploy.json
+      Push-AppveyorArtifact deploy.json
+      # Execute the deploy script on the NV Access server via ssh.
+      # Warning: if the server address is changed, 
+      # The new address must be also included in appveyor\ssh_known_hosts in this repo
+      # Otherwise ssh will freeze on input!
+      cat deploy.json | ssh nvaccess@deploy.nvaccess.org nvdaAppveyorHook
 
       # Upload symbols to Mozilla.
       py -m pip install --no-warn-script-location requests
@@ -267,6 +288,9 @@ deploy_script:
 on_finish:
   # add a message to point to the NVDA build, to make testing PR's easier.
   - ps: |
+      # Save an exact list of the actual Python packages and their versions that got installed, to aide in reproducing a build
+      .\venvUtils\exportPackageList.bat installed_python_packages.txt
+      Push-AppveyorArtifact installed_python_packages.txt
       $appVeyorUrl = "https://ci.appveyor.com"
       $exe = Get-ChildItem -Name output\*.exe
       if($?){

appveyor/ssh_known_hosts

@@ -1,2 +1 @@
-exbi.nvaccess.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJZYcI3sI+IbIaiUcd5/LIL5+wfEoO+NQ8Gw1Ww9bxwpOyP/HYu/1yzuRNTDSckGM3hqaGGliTW2/3DYOvPDUwI=
-45.33.21.113 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJZYcI3sI+IbIaiUcd5/LIL5+wfEoO+NQ8Gw1Ww9bxwpOyP/HYu/1yzuRNTDSckGM3hqaGGliTW2/3DYOvPDUwI=
+deploy.nvaccess.org,45.33.23.174 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBZdYWLmrUzPukq3wa15XFFgi2THBc92S9gpvZwrDUDkxNUD5VzRISf2uuFD+cRwn+gwj2jvH77fCFtwqfo4N3w=

appx/manifest.xml.subst

@@ -24,7 +24,7 @@
 		<TargetDeviceFamily 
 			Name="Windows.Desktop" 
 			MinVersion="10.0.18990.0" 
-			MaxVersionTested="10.0.18995.0" 
+			MaxVersionTested="10.0.19043.0" 
 		/>
 	</Dependencies>
 	<Capabilities>