What's new and readme: we are moving to Python 3.7 by josephsl · Pull Request #9942 · nvaccess/nvda

josephsl · 2019-07-17T02:34:02Z

Link to issue number:

Fixes Update changes file on threshold branch #9862
Gives closure to Upgrade NVDA codebase to Python 3 #7105

Summary of the issue:

What's new and readme still mentions Python 2.7, and in case of what's new, changes were needed.

Description of how this pull request fixes the issue:

Edits include mentioning Python 3.7 in both readme and what's new, as well as fixing things here and there in what's new/Threshold section. See the below section regarding work included in what's new and readme through this pull request and other Threshold work.

Testing performed:

None

Known issues with pull request:

Some parts are missing, such as filling in which issue introduced support for pyserial 3.4 and other things. What's new and readme are subject to change based on comments.

Change log entry:

None

Pull requests addressed by this work

Items with a checkmark are mentioned in what's new. Merge messages are ignored.

Speech refactor

New speech framework including callbacks, beeps, sounds, profile switches and prioritized queuing (New speech framework including callbacks, beeps, sounds, profile switches and prioritized queuing #7599)

Python 3.7

No entry necessary

Fixes for BrailliantB and FreedomScientific (PR Fixes for BrailliantB and FreedomScientific #9839)
Python 2/3: relative imports (Python 2/3: relative imports #9670)
Python 3: print statement -> print function (Python 3: print statement -> print function #9664)
Python 3: file function -> open function (Python 3: file function -> open function #9663)
Python 3: use floor division (//) to obtain integer results from division operations (Python 3: use floor division (//) to obtain integer results from division operations #9692)
Python 3: use importlib for importing modules (Python 3: use importlib for importing modules #9693)
Python 3: dictionary iteration/list methods made compatible with Python 3 (Python 3: dictionary iteration/list methods made compatible with Python 3 #9671)
Python 3: xrange -> range (Python 3: xrange -> range #9627)
SAPI 5 synth driver: percentage to pitch/rate must return an integer. Re Python 3: division operator differences between classic and the new division #9641. (SAPI 5: percent to rate/pitch must return an integer #9699)
Python 3: except exception, e -> except as e (Python 3: except exception, e -> except as e #9698)
Python 3: dict.haskey -> key in dict (Python 3: dict.haskey -> key in dict #9697)
Check pot: file -> open (Check pot: file -> open #9718)
WinConsole handler/Python 3: use floor divisoin assignment operator. Re Python 3: division operator differences between classic and the new division #9641. (Windows Console Handler: use floor division assignment operator when calculating coordinates #9737)
Unittests/Python 3: update syntax, catch unbound method registration test in extension points as expected failure (Unittests/Python 3: update syntax, catch unbound method registration test in extension points as expected failure #9726)
Python 3: zip function -> list(zip), itertools.izip -> zip() (Python 3: zip function -> list(zip), itertools.izip -> zip() #9729)
Global commands: restore commands affected by division operator differences between Python 2 and 3 (Global commands: restore commands affected by division operator differences between Python 2 and 3 #9748)
Python 3: more native imports and modules (Python 3: more native imports and modules #9751)
Input core/input gestures dialog: script function object has no attribute func. (Input core/input gestures dialog: script function object has no attribute named __func__ #9752)
Python 3: use native metaclass definition syntax (Python 3: use native metaclass definition syntax #9753)
System tests/Python 3: syntax and imports (System tests/Python 3: syntax and imports #9801)
App module handler/does app mod exists: return false when called before imports are initialized (App module handler/does app mod exists: return false when called before imports are initialized #9800)
Add-on handler: ugettext -> gettext (Add-on handler/translations: ugettext -> gettext #9810)
Python 3/urllib: imports, timeout (Python 3/urllib: imports, timeout #9820)
Update check/Python 3: update check resource is encoded as bytes, thus make it Unicode. Re Update check/Python 3: "bytes-like object is required" traceback when checking for updates #9819. (Update check/Python 3: convert update check resource from bytes to Unicode #9837)
WinKernel: create a wrapper for MoveFileExW function and associated flag constants (PR WinKernel: create a wrapper for MoveFileExW function and associated flag constants #9849)
Python 3: use UTF-8 decoding for font name with Scintilla controls (PR Scintilla: use UTF-8 decoding to convert font name to Unicode #9893)
Python 3: MoveFileEx - catch WindowsError instead of WinError (PR MoveFileEx: catch WindowsError instead of WinError #9876)
Python 3: Use built-in "input" instread of "raw_input" (PR NvDA slave: raw_input -> input #9908)
Python 3: Check for None during check for previous installation version in Installer GUI (PR Installer GUI/Python 3: check if install comparison method returns None or not #9924)
Update setup script for Python 3.7 (Update setup script for Python 3.7 #9605)
Update Scons build environment to run on Python 3 (Update Scons build environment to run on Python 3 #9667)
Python 3: Fix use of subprocess (Python 3: Fix use of subprocess #9716)
Python 3: No longer encode or decode if it is not necessary (Python 3: No longer encode or decode if it is not necessary #9734)
Python 3: fix several remaining division errors (Python 3: fix several remaining division errors #9774)
Write syntax errors and tracebacks to python console (Python 3: Write syntax errors and tracebacks to python console #9767)
Python 3: fix logging level not being updated correctly (Python 3: fix logging level not being updated correctly #9809)
Python 3: Fix deprecation warning extensionPoints.callWithSUpportedKwargs (PR Python 3: work around deprecation warning with a major rewrite of extensionPoints.callWithSUpportedKwargs, and add some tests for Py3 specific functionality #9778)
Fix comparison error in start menu (PR Python 3: Fix comparison error in start menu #9846)
Update BRLTTY driver (PR Python 3: Update BRLTTY driver #9821)
Python 3 one core: Don't check for sys.frozen in the check method. (PR Python 3 one core: Don't check for sys.frozen in the check method. #9857)
Python 3: fix say all for DisplayModelEditableText (PR Python 3: fix say all for DisplayModelEditableText #9866)
Python 3: sys.exec_prefix > sys.prefix for nvda_slave (PR Python 3: sys.exec_prefix > sys.prefix for nvda_slave #9871)
Python 3: Fix usages of "open" function (PR Python 3: Fix several instances of open function usage #9853)
Python3: Fix save scripts in lockapp (PR Python 3: Fix SAFE_SCRIPTS in lockapp appModule #9884)
Python 3: When processing xml coming from virtual buffers, make sure that surrogate characters are handled properly (PR Python 3: When processing xml coming from virtual buffers, make sure that surrogate characters are handled properly #9897)
Python 3: Remove many instances of "py3 review required" comments (PR Remove many instances of "py3 review required" comments #9890)
Python 3: Fix adobe reader int/str to NoneType comparisons as well as other identified cases (PR: Fix adobe reader int/str to NoneType comparisons as well as other identified cases #9881)
Revert "Restore VoiceSettingsSlider to ensure we don't break compatibility with Audio Themes add-on" (PR Threshold: Revert "Restore VoiceSettingsSlider to ensure we don't break compatibility with Audio Themes add-on (#9826)" #9907)
Python 3: Clean the current words buffer in more cases (PR Python 3: Clean the current words buffer in more cases #9877)
Python 3: Remove Python 2 specific workarounds (PR Remove Python 2 specific workarounds from Python 3 code #9889)
Python 3: Fix loading of espeak variants (PR Python 3: Fix wrong indent level causing loading of espeak variants to fail #9920)
Python 3: Fix several local overrides of range function (PR Python 3: Fix several local overrides of range function #9912)
Fix waiting for outlook/powerpoint window to exist for ever (PR Fix waiting for outlook/powerpoint window to exist for ever #9950)
Log never handled speech indexes, and don't make the speech queue lag behind (PR Log never handled speech indexes, and don't make the speech queue lag behind #9946)
Fix error in JAVA text ranges (PR Python 3: Fix error when fetching JAVA text ranges #9951)
appveyor: build snapshots for threshold
OneCore synthDriver: fix up merge conflicts.
Only clear curWordChars on focus / caret movement (Only clear curWordChars on focus / caret movement #9597)
import Python 3 native modules (import Python 3 native modules #9703)
Python 3: sys.maxint -> sys.maxsize, as maxint is gone in Python 2. Re Upgrade NVDA codebase to Python 3 #7105. (Python3: sys.maxint to sys.maxsize #9704)
Python3: os.getcwdu -> os.getcwd (Python3: os.getcwdu -> os.getcwd #9705)
Python 3/remote python console: SocketServer -> socketserver. (Python 3/remote python console: SocketServer -> socketserver. #9706)
Update miscDeps to latest master, containing python3 txt2tags.
Python 3: the thread module is no longer available. Use threading instead. (Python 3: use threading instead of thread. #9713)
Python3: fix remaining syntax errors found by py2exe (Python3: fix remaining syntax errors found by py2exe #9719)
Python3: fix NVDA's logging infrastructure (Python3: fix NVDA's logging infrastructure #9712)
Python3: gettext is unicode, and pgettext is placed in builtins properly (Python3: gettext is unicode, and pgettext is placed in builtins properly #9714)
objidl: HRESULT is no longer available in ctypes.wintypes. Get it from comtypes. (Python3: import HRESULT from comtypes instead of ctypes.wintypes #9745)
remove use of unichr, basestring and unicode objects. (remove use of unichr, basestring and unicode objects. #9724)
Remove need for hashing NVDAObject and TextInfo objects (Remove need for hashing NVDAObject and TextInfo objects #9746)
Revert "Remove need for hashing NVDAObject and TextInfo objects (Remove need for hashing NVDAObject and TextInfo objects #9746)" (Revert "Remove need for hashing NVDAObject and TextInfo objects" #9756)
Python3: base virtualBuffer: coerce attribute values to a string before escaping for regular expression (Python3: base virtualBuffer: coerce attribute values to a string before escaping for regular expression #9759)
cmp no longer exists in Python3. Replace with (a>b)-(a<b) (Remove usage of cmp function #9758)
Python3: Provide hash methods where ever we have provided eq methods (Python3: Provide __hash__ methods where ever we have provided __eq__ methods #9757)
Correct a few places where we are trying to do integer comparisons with None. (Correct a few places where we are trying to do integer comparisons with None #9794)
Ensure NVDA contains a signed and valid python37.dll (Ensure NVDA contains a signed and valid python37.dll #9796)
Python3: implement new way of converting COMError to CallCancelled (Python3: implement new way of converting COMError to CallCancelled #9795)
bdDetect: change basestring to str. This was the only thing causing NVDA not to run in this branch, and will be introduced exactly the same by pr Python3 convert braille drivers #9736 anyway.
Make output of both py2exe and nose more verbose to aide in future build issues.
Revert "App module handler/does app mod exists: return false when called before imports are initialized (App module handler/does app mod exists: return false when called before imports are initialized #9800)" (Revert "App module handler/does app mod exists: return false when called before imports are initialized " #9815)
config.addConfigDirsToPythonPackagePath: When adding scratchpad dirs, create them if they don't already exist. (Ensure scratchpad dirs exist before adding them to module paths #9814)
Update miscDeps to contain new brlApi module (Update miscDeps to contain new brlApi module #9812)
Python3: stop using unichr in brailleInput.py (Python3: stop using unichr in brailleInput.py #9835)
Python3 convert braille drivers (PR Python3 convert braille drivers #9736)
Fix buildversion for threshold branch
Fix api back compat version number
Python 3: Fix missing import, remove unused imports (PR Fix missing import, remove unused imports #9901)
Python 3: Fix error with accPropServer (PR Python 3: Fix error with accPropServer #9900)
txt2tags for python3 Python 3: find a suitable and compatible replacement for txt2tags #8734 (txt2tags for python3 #8734 #9648)
Change iter.next() to next(iter) (PR Change iter.next() to next(iter) #9535)
Remove .pyc and .pyo files when updating (PR Threshold: "SyntaxError: invalid or missing encoding declaration for 'brailleDisplayDrivers\\handyTech.pyc'" #9960)
Python3: Fix SAPI5 and MSSP on installed NVDA (PR Fix sapi5 and MSSP on installed copies of threshold #9964)
Detect caret movement using events, except for IA2Web Detect caret movement using events, except for IA2Web #9933

josephsl · 2019-07-17T02:35:18Z

Hi all,

Comments from everyone who has worked on or commented on Python 3 are appreciated, especially from Reef, @jcsteh, and @LeonarddeR.

Thanks,

josephsl · 2019-07-17T03:59:33Z

Hi, right, thanks for catching this, which also had a grammar problem (will resolve it once I wake up from sleep in the morning). Thanks.

LeonarddeR

You can change the threshold release heading to 2019.3

josephsl · 2019-07-17T05:38:49Z

Hi, I knew there was an issue about mentioning minor versions in docs – thanks for the link to the issue. Thanks.

josephsl · 2019-07-17T14:23:04Z

Hi,

Should we mention textUtils?

Thanks.

LeonarddeR · 2019-07-17T14:59:42Z

yes, sounds like a good idea. Probably best to point at examples from the changelog.

josephsl · 2019-07-17T22:24:50Z

Hi,

Regarding text utils, I have nothing to say - can't come up with a descriptive changelog for developers at the moment.

Anything else from our PR pool? If not, I'll switch the PR to review mode.

Thanks.

LeonarddeR · 2019-07-18T04:58:28Z

How about something within the lines of:

Added a new textUtils module that makes it easier to deal with string differences between Python 3 strings and Windows unicode strings. See the module documentation and textInfos.offsets module for example implementations. (textUtils module to deal with offset differences between Python 3 strings and Windows wide character strings with surrogate characters #9545)

josephsl · 2019-07-18T12:21:57Z

Hi,

Text utils: done.

Thanks.

feerrenrut

This is looking pretty good. Please update the description to cover which PR's (to threshold / threshold staging) have been covered by this. You can get most of the way there by copying entries from git log on the threshold branch. I suggest using the checkbox markdown:

- [ ] some PR not covered
- [x] some PR covered

josephsl · 2019-07-22T16:28:36Z

Hi,

I'll use git shortlog to edit the description.

Thanks.

josephsl · 2019-07-23T02:21:17Z

Hi,

To @LeonarddeR: whenever you are ready, please take a look at this PR. Thanks.

feerrenrut · 2019-07-23T12:25:20Z

Thanks @josephsl, I have updated the description so that they render as a checkboxes. It looks like there are still many PR's that need consideration.

josephsl · 2019-07-23T13:45:39Z

Hi, yes, including the just merged one (#9960). Thanks.

feerrenrut · 2019-07-24T10:56:05Z

If no one volunteers before tomorrow (25-July 0600 UTC+0) , then I'll finish this off.

LeonarddeR

I found two sentences where has should be have IMO. Could be wrong for the latter, though.

LeonarddeR · 2019-07-24T11:38:42Z

- config.ConfigManager.getConfigValidationParameter is removed. A bug in getConfigValidation has been fixed in order for it being able to fully replace getConfigValidationParameter
- inputCore.InputGesture.logIdentifier property has been removed. Now use inputCore.InputGesture.identifiers[0] instead.
+- textInfos.Point and textInfos.Rect has been removed and replaced by locationHelper.Point and locationHelper.RectLTRB, respectively.
+- braille.BrailleHandler._get/set_tether has been removed. Use getTether and setTether respectively.


Suggested change

- braille.BrailleHandler._get/set_tether has been removed. Use getTether and setTether respectively.

- braille.BrailleHandler._get/set_tether have been removed. Use getTether and setTether respectively.

josephsl · 2019-07-24T20:18:28Z

Hi, I think part of that may have to do with subtle English grammar differences.

josephsl · 2019-07-24T21:06:08Z

Hi,

Let me at least commit the requested edits, and then I think we should decide our next steps.

Thanks.

feerrenrut · 2019-07-25T14:04:29Z

I'd like to get this finished off and merged in, are you going to continue with it @josephsl? I don't think I will hold back threshold any longer.

josephsl · 2019-07-25T14:06:26Z

Hi, feel free to merge – let’s get the party started. Thanks.

josephsl · 2019-07-25T18:31:10Z

Hi,

Clarification: I'm done with this one.

Thanks.

Cherry picked changes from #9942 Closes #8421

LeonarddeR · 2019-07-27T10:53:35Z

@josephsl: could you please do a base branch switch for this?

Because Appveyor changes Python x.y.z fromtime to time, it is bettre to just say Python x.y.

What's new entry suggested by Leonard de Ruijter (Babbage).

Reviewed by Leonard de Ruijter (Babbage): has -> have for two entries.

josephsl · 2019-07-27T15:46:03Z

Done.

derekriemer · 2019-08-04T21:47:50Z

Should this be merged?

josephsl · 2019-08-04T21:52:43Z

Hi, when it is ready, yes. Thanks.

feerrenrut · 2019-08-05T12:10:11Z

@derekriemer The checklist for PR's needs to be completed, I suspect many of these do not need an entry. This is on my list of things to do, but hasn't yet received priority.

feerrenrut · 2019-08-05T20:18:56Z

I have gone through the unchecked items and added entries.

I also re-arranged the current items in the changes file.

Changes were addressed

josephsl requested a review from feerrenrut July 17, 2019 02:34

XLTechie reviewed Jul 17, 2019

View reviewed changes

Comment thread user_docs/en/changes.t2t Outdated

LeonarddeR suggested changes Jul 17, 2019

View reviewed changes

Comment thread readme.md Outdated

Comment thread user_docs/en/changes.t2t Outdated

Comment thread user_docs/en/changes.t2t Outdated

josephsl marked this pull request as ready for review July 17, 2019 23:39

feerrenrut approved these changes Jul 22, 2019

View reviewed changes

LeonarddeR previously requested changes Jul 24, 2019

View reviewed changes

feerrenrut mentioned this pull request Jul 25, 2019

Migrate NVDA to Python 3 #9974

Merged

feerrenrut added a commit that referenced this pull request Jul 25, 2019

Readme now refers to requirement for Python 3

23ad41e

Cherry picked changes from #9942 Closes #8421

josephsl added 3 commits July 27, 2019 08:38

What's new: remove extraneous indentation.

27fac42

What's new: add missing punctuation.

39a625b

What's new: say hello to Python 3.7! Re nvaccess#7105

62ad9b9

josephsl added 3 commits July 27, 2019 08:39

What's new: various gramar fixes.

f47083c

What's new and readme: Python 3.7.4 -> 3.7. Re nvaccess#8421.

d3c2d10

Because Appveyor changes Python x.y.z fromtime to time, it is bettre to just say Python x.y.

What's new: more grammar fixes.

b4f1c60

josephsl changed the base branch from threshold to changesFileLineEndingsChanged_master July 27, 2019 15:42

josephsl changed the base branch from changesFileLineEndingsChanged_master to master July 27, 2019 15:43

josephsl added 3 commits July 27, 2019 08:44

What's new: add textUtils entry. Re nvaccess#9545.

7e4672d

What's new entry suggested by Leonard de Ruijter (Babbage).

What's new: grammar fixes. Re nvaccess#9918.

2c2752c

What's new: address grammar edit comments.

fc7e441

Reviewed by Leonard de Ruijter (Babbage): has -> have for two entries.

josephsl force-pushed the py3_docsAndReadme branch from 1cd5b05 to fc7e441 Compare July 27, 2019 15:45

Merged master

ae317e8

feerrenrut added 2 commits August 5, 2019 20:32

Merge remote-tracking branch 'origin/master' into HEAD

50d0f02

Remaining what's new changes for threshold

d3299bc

feerrenrut merged commit ddf27fa into nvaccess:master Aug 5, 2019

nvaccessAuto added this to the 2019.3 milestone Aug 5, 2019

josephsl deleted the py3_docsAndReadme branch September 16, 2019 01:57

	- braille.BrailleHandler._get/set_tether has been removed. Use getTether and setTether respectively.
	- braille.BrailleHandler._get/set_tether have been removed. Use getTether and setTether respectively.

Uh oh!

Uh oh!

Conversation

josephsl commented Jul 17, 2019 • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Link to issue number:

Summary of the issue:

Description of how this pull request fixes the issue:

Testing performed:

Known issues with pull request:

Change log entry:

Pull requests addressed by this work

Speech refactor

Python 3.7

No entry necessary

Uh oh!

josephsl commented Jul 17, 2019

Uh oh!

Uh oh!

josephsl commented Jul 17, 2019 via email • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LeonarddeR left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

josephsl commented Jul 17, 2019 via email • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

josephsl commented Jul 17, 2019

Uh oh!

LeonarddeR commented Jul 17, 2019 via email

Uh oh!

josephsl commented Jul 17, 2019

Uh oh!

LeonarddeR commented Jul 18, 2019

Uh oh!

josephsl commented Jul 18, 2019

Uh oh!

feerrenrut left a comment

Choose a reason for hiding this comment

Uh oh!

josephsl commented Jul 22, 2019

Uh oh!

josephsl commented Jul 23, 2019

Uh oh!

feerrenrut commented Jul 23, 2019

Uh oh!

josephsl commented Jul 23, 2019 via email • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

feerrenrut commented Jul 24, 2019

Uh oh!

LeonarddeR left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

LeonarddeR Jul 24, 2019

Choose a reason for hiding this comment

Uh oh!

josephsl commented Jul 24, 2019 via email • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

josephsl commented Jul 24, 2019

Uh oh!

feerrenrut commented Jul 25, 2019

Uh oh!

josephsl commented Jul 25, 2019 via email • edited by feerrenrut Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

josephsl commented Jul 25, 2019

Uh oh!

LeonarddeR commented Jul 27, 2019

Uh oh!

josephsl commented Jul 27, 2019

Uh oh!

derekriemer commented Aug 4, 2019

Uh oh!

josephsl commented Jul 17, 2019 •

edited by feerrenrut

Loading

josephsl commented Jul 17, 2019 via email •

edited by feerrenrut

Loading

josephsl commented Jul 17, 2019 via email •

edited by feerrenrut

Loading

josephsl commented Jul 23, 2019 via email •

edited by feerrenrut

Loading

josephsl commented Jul 24, 2019 via email •

edited by feerrenrut

Loading

josephsl commented Jul 25, 2019 via email •

edited by feerrenrut

Loading

josephsl commented Aug 4, 2019 via email •

edited by feerrenrut

Loading